s1monw commented on a change in pull request #1909:
URL: https://github.com/apache/lucene-solr/pull/1909#discussion_r493482012
##########
File path: lucene/core/src/java/org/apache/lucene/index/SortingCodecReader.java
##########
@@ -510,4 +457,52 @@ public LeafMetaData getMetaData() {
return metaData;
}
+ // we try to cache the last used DV or Norms instance since during merge
+ // this instance is used more than once. We could in addition to this single
instance
+ // also cache the fields that are used for sorting since we do the work
twice for these fields
+ private String cachedField;
+ private Object cachedObject;
+ private boolean cacheIsNorms;
+
+ private <T> T getOrCreateNorms(String field, IOSupplier<T> supplier) throws
IOException {
+ return getOrCreate(field, true, supplier);
+ }
+
+ @SuppressWarnings("unchecked")
+ private synchronized <T> T getOrCreate(String field, boolean norms,
IOSupplier<T> supplier) throws IOException {
+ if ((field.equals(cachedField) && cacheIsNorms == norms) == false) {
+ assert assertCreatedOnlyOnce(field, norms);
+ cachedObject = supplier.get();
+ cachedField = field;
+ cacheIsNorms = norms;
+
+ }
+ assert cachedObject != null;
+ return (T) cachedObject;
+ }
+
+ private final Map<String, Integer> cacheStats = new HashMap<>(); // only
with assertions enabled
+ private boolean assertCreatedOnlyOnce(String field, boolean norms) {
+ assert Thread.holdsLock(this);
+ // this is mainly there to make sure we change anything in the way we
merge we realize it early
+ Integer timesCached = cacheStats.compute(field + "N:" + norms, (s, i) -> i
== null ? 1 : i.intValue() + 1);
+ if (timesCached > 1) {
+ assert norms == false :"[" + field + "] norms must not be cached twice";
Review comment:
I think what we do here is we pull the already merged norms instance
from disk instead of the one from the source reader. Is that what you mean in
`PushPostingsWriterBase`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
