GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/527
LUCENE-8609: Allow getting consistent docstats from IndexWriter
Today we have #numDocs() and #maxDoc() on IndexWriter. This is enough
to get all stats for the current index but it's subject
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/526
@mikemccand I pushed changes. I think it's ready
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/526
LUCENE-8608: Extract utility class to iterate over terms docs
Today we re-implement the same algorithm in various places
when we want to consume all docs for a set/list of terms
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/523
LUCENE-8602: Share TermsEnum if possible while applying DV updates
Today we pull a new terms enum when we apply DV updates even though the
field stays the same which is the common case
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/522
@jpountz I will get this in as is and then explore what we can do as a
followup with DocIdSetBuilder. We do need additional stats to make efficient
use of it IMO and this already yields
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/522
LUCENE-8599: Use sparse bitset to store docs in
SingleValueDocValuesFieldUpdates
Using a sparse bitset in SingleValueDocValuesFieldUdpates allows storing
which documents have an update much
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/521
> t's interesting that you found out that most of the time is spent with
packed ints when sorting, likely because of swaps: the sorting impl that is
being used (InPlaceMergeSorter) is the i
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/521#discussion_r240234021
--- Diff:
lucene/core/src/java/org/apache/lucene/index/NumericDocValuesFieldUpdates.java
---
@@ -53,14 +55,28 @@ BytesRef binaryValue
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/521#discussion_r240047328
--- Diff: lucene/core/src/test/org/apache/lucene/index/Benchmark.java ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/521
LUCENE-8598: Improve field updates packed values
DocValuesFieldUpdats are using compact settings for packet ints that causes
dramatic slowdowns when the updates are finished and sorted
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/517
LUCENE-8595: Fix interleaved DV update and reset
This change fixes a bug where interleaved update and reset value
to the same doc in the same updates package looses an update
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/516
@jpountz I pushed changes
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/516#discussion_r239409024
--- Diff:
lucene/backward-codecs/src/test/org/apache/lucene/index/TestBackwardsCompatibility.java
---
@@ -1587,6 +1590,76 @@ public void
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/516#discussion_r239408501
--- Diff:
lucene/backward-codecs/src/test/org/apache/lucene/index/TestBackwardsCompatibility.java
---
@@ -1587,6 +1590,76 @@ public void
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/516#discussion_r239408275
--- Diff:
lucene/backward-codecs/src/test/org/apache/lucene/index/TestBackwardsCompatibility.java
---
@@ -1587,6 +1590,76 @@ public void
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/516
this was found in https://github.com/elastic/elasticsearch/pull/36286
---
-
To unsubscribe, e-mail: dev-unsubscr
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/516
LUCENE-8594: DV update are broken for updates on new field
A segmemnt written with Lucene70Codec failes if it ties to update
a DV field that didn't exist in the index before it was upgraded
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/515
LUCENE-8593?: Specialize single value numeric DV updates
The case when all values are the the same on a numeric field update
is common for soft_deletes. With the new infrastucture
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239136654
--- Diff:
lucene/core/src/java/org/apache/lucene/index/FieldUpdatesBuffer.java ---
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/513
> Any sense of how much of RAM reduction this is over current impl?
I ran a very simple measurement on top of `BufferedUpdates` adding 10k
random updates with a constant seed. H
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239062234
--- Diff:
lucene/core/src/java/org/apache/lucene/index/FieldUpdatesBuffer.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239062190
--- Diff:
lucene/core/src/java/org/apache/lucene/index/FieldUpdatesBuffer.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239036539
--- Diff:
lucene/core/src/java/org/apache/lucene/index/FieldUpdatesBuffer.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239036461
--- Diff:
lucene/core/src/java/org/apache/lucene/index/FieldUpdatesBuffer.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/513#discussion_r239036263
--- Diff: lucene/core/src/java/org/apache/lucene/index/BufferedUpdates.java
---
@@ -288,15 +186,24 @@ void clear() {
deleteTerms.clear
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/513
@mikemccand can you take a look?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/513
LUCENE-8590: Optimize DocValues update datastructures
Today we are using a LinkedHashMap to buffer doc-values updates in
BufferedUpdates. This on the one hand uses an Object based
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/503
@msokolov I measures a pretty signficant lock contention on this lock
without my change. with the change it went down significantly. Refreshes got
30% faster for that benchmark (small docs
Github user s1monw closed the pull request at:
https://github.com/apache/lucene-solr/pull/503
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/503
pushed
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/503
I recommend to look at the diff with `?w=1` ->
https://github.com/apache/lucene-solr/pull/503/files?w=1
---
-
To unsubscr
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/503
@mikemccand can you take a look
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/503
LUCENE-8571: Don't block on FrozenBufferedUpdates#apply during
IW#processEvents
While indexing we try to apply frozen deletes packages concurrently
on indexing threads if necessary
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/502#discussion_r235273221
--- Diff: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java ---
@@ -4398,42 +4426,41 @@ private int mergeMiddle(MergePolicy.OneMerge merge
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/502
@mikemccand can you take a look at this?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/502
LUCENE-8569: Never count soft-deletes if reader has no hard-deletes
Today we count the actual soft-deletes during a merge which is
unnecessary if there are no hard-deletes present
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/480
@jimczi pushed some changes
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/480
LUCENE-8535: Drop out of the box Block-Join highlight support
Highlighter doesn't support ToParent and ToChildBlockJoinQuery out of the
box anymore. In oder to highlight on Block-Join
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/454
LUCENE-8503: Simplify unwrapping Filter*Reader
Today we have 3 different kinds of FilterIndexReader. While
FilterDirecotryReader
and FilterLeafReader are simple to distinguish
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/453
LUCENE-8458: Allow access to delegate in FilterCodecReader
FilterCodecReader doesn't allow access to it's delegate like other
filter readers. This adds a new getDelegate method to access
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/445
LUCENE-8484: Allow SubReaderWrapper to filter out entire readers
Today we can only wrap readers in SubReaderWrapper but never filter them out
entirely. This causes a invariant for soft
Github user s1monw commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/372#discussion_r186633149
--- Diff:
solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java
---
@@ -619,6 +619,6 @@ private void assertEquals(long numOrds
42 matches
Mail list logo
