Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21700#discussion_r200249732
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/streaming/state/BoundedSortedMap.java
---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.streaming.state;
+
+import java.util.Comparator;
+import java.util.Map;
+import java.util.TreeMap;
+
+/**
+ * This class implements bounded {@link java.util.SortedMap} based on
{@link java.util.TreeMap}.
+ *
+ * As TreeMap does, this implementation sorts elements in natural order,
and cuts off
+ * smaller elements to retain at most bigger N elements.
+ *
+ * You can provide reversed order of comparator to retain smaller elements
instead.
+ *
+ * This class is not thread-safe, so synchronization would be needed to
use this concurrently.
+ *
+ * @param <K> key type
+ * @param <V> value type
+ */
+public class BoundedSortedMap<K, V> extends TreeMap<K, V> {
+
+ private final int limit;
+
+ /**
+ * Constructor
+ *
+ * @param comparator comparator instance to compare between keys
+ * @param limit bounded size
+ */
+ public BoundedSortedMap(Comparator<K> comparator, int limit) {
+ super(comparator);
+ this.limit = limit;
+ }
+
+ @Override
+ public void putAll(Map<? extends K, ? extends V> map) {
+ for (Map.Entry<? extends K, ? extends V> entry : map.entrySet()) {
--- End diff --
Thanks for the great suggestion. While we can't assume that map's type is
SortedMap, looks like we could check the type of map in runtime and apply your
suggestion. Will apply it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
