On 9/2/2019 9:19 AM, Erick Erickson wrote:
Anyway, it occurred to me that once a max-sized segment is created, _if_ we
write the segments_n file out with the current state of the index, we could
freely delete the segments that were merged into the new one. With 300G indexes
(which I see
Yeah, if merging down to 1 segment, there’s no choice in the matter. But with
the changes to TMP in 7.5, users have to explicitly use maxSegments=1 to get
that behavior.
It also seems we could usefully predict how much free space we need, each merge
thread does have a list of segments, can
I think this is worth exploring?
Essentially, after each large merge, we'd need to 1) commit, and 2) refresh
any open readers (and close the old readers), to fully free up transient
disk usage. Maybe we could somehow track the current transient extra disk
usage of the index + open readers and
Thanks for offering!
I will give you access but I believe you need to create an account first.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Fri, Sep 13, 2019 at 1:15 PM Bram Van Dam wrote:
> Hey folks,
>
> I'd like to request write access to
[
https://issues.apache.org/jira/browse/SOLR-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929379#comment-16929379
]
Michael Gibney commented on SOLR-13094:
---
It looks like there may be other things going on here, but
Hey folks,
I'd like to request write access to the Solr cwiki. Mostly so I can
correct the odd typo as I see them.
Thanks,
- Bram
[
https://issues.apache.org/jira/browse/SOLR-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man reopened SOLR-13622:
-
Uwe's jenkins servers weren't being included in my reports for over a month –
that's why you didn't see any
On 02/09/2019 17:19, Erick Erickson wrote:
> 4> Don’t quite know what to do if maxSegments is 1 (or other very low number).
Having maxSegments set to > 5 (or whatever) seems like an acceptable
constraint if it enables optimize without 200% disk usage.
> Something like this would also pave the
[
https://issues.apache.org/jira/browse/SOLR-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley resolved SOLR-13739.
-
Fix Version/s: 8.3
Assignee: David Smiley
Resolution: Fixed
Thanks for
[
https://issues.apache.org/jira/browse/SOLR-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929349#comment-16929349
]
ASF subversion and git services commented on SOLR-13739:
Commit
[
https://issues.apache.org/jira/browse/SOLR-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929347#comment-16929347
]
ASF subversion and git services commented on SOLR-13739:
Commit
dsmiley closed pull request #855: SOLR-13739: Improve performance on huge
schema updates
URL: https://github.com/apache/lucene-solr/pull/855
This is an automated message from the Apache Git Service.
To respond to the
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929317#comment-16929317
]
Steve Rowe commented on LUCENE-8951:
bq. I will take care of ASF Jenkins after lunch. I am not sure
[
https://issues.apache.org/jira/browse/LUCENE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929314#comment-16929314
]
Michael McCandless commented on LUCENE-8939:
Thanks [~jpountz], this is an exciting
[
https://issues.apache.org/jira/browse/LUCENE-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929311#comment-16929311
]
David Smiley commented on LUCENE-8921:
--
Barring further feedback, I'll commit what Bruno has
[
https://issues.apache.org/jira/browse/LUCENE-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-8975.
--
Fix Version/s: 8.3
Resolution: Fixed
> Code Cleanup: Use entryset for map iteration
[
https://issues.apache.org/jira/browse/LUCENE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929307#comment-16929307
]
Adrien Grand commented on LUCENE-8939:
--
Thanks [~mikemccand] I had lost track of this, I agree it
[
https://issues.apache.org/jira/browse/LUCENE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-8939.
--
Fix Version/s: 8.3
Resolution: Fixed
> Shared Hit Count Early Termination
>
[
https://issues.apache.org/jira/browse/LUCENE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929306#comment-16929306
]
ASF subversion and git services commented on LUCENE-8939:
-
Commit
[
https://issues.apache.org/jira/browse/LUCENE-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929305#comment-16929305
]
ASF subversion and git services commented on LUCENE-8939:
-
Commit
[
https://issues.apache.org/jira/browse/LUCENE-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929280#comment-16929280
]
Atri Sharma commented on LUCENE-8978:
-
Both the runs are for wikimedium2m with concurrent searches
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531276346
Hmm, the precommit failure seems to be coming from `JdbcDataSource.java`?
[
https://issues.apache.org/jira/browse/LUCENE-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929277#comment-16929277
]
Atri Sharma commented on LUCENE-8978:
-
Run with propagating global minimum scores
||Task
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531275454
@jimczi Here is the link:
iverase commented on issue #627: LUCENE-8746: Make EdgeTree (aka ComponentTree)
support different type of components
URL: https://github.com/apache/lucene-solr/pull/627#issuecomment-531271483
See #878
This is an automated
iverase closed pull request #627: LUCENE-8746: Make EdgeTree (aka
ComponentTree) support different type of components
URL: https://github.com/apache/lucene-solr/pull/627
This is an automated message from the Apache Git
iverase opened a new pull request #878: LUCENE-8746: Refactor EdgeTree
URL: https://github.com/apache/lucene-solr/pull/878
Another try in refactoring edge tree. This PR splits Edge Tree class into
two and adds a new interface:
* Component2D: Interface defining an object that knows
jimczi commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531266829
Can you run another benchmark now that we propagate the global min score to
the scorer ?
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531261223
@jimczi Updated
This is an automated message from the Apache
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324221245
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324218422
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324218620
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
[
https://issues.apache.org/jira/browse/SOLR-13758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929237#comment-16929237
]
Erick Erickson commented on SOLR-13758:
---
Do you have specific cases you'd like to see? Patches
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324216889
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929225#comment-16929225
]
Uwe Schindler commented on LUCENE-8951:
---
OK, all jobs changed by them with "Configuration Slicer"
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929222#comment-16929222
]
Uwe Schindler commented on LUCENE-8951:
---
I am on INFRA Slack if we can use the configuartion
atris commented on issue #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876#issuecomment-531233493
Thanks!
This is an automated message from the
msokolov commented on issue #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876#issuecomment-531233209
thanks! merged
This is an automated message
msokolov merged pull request #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876
This is an automated message from the Apache Git Service.
To
atris commented on issue #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876#issuecomment-531230646
Thanks, missed the camel casing, fixed now
This
msokolov commented on issue #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876#issuecomment-531229416
I'll commit it. As I looked at this I couldn't help noticing a little
spelling nit; we have {{thresHold}} but
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531228068
@jimczi Updated, please see.
One thing we could do is cache the bottom docID (doc + docBase) in
BottomValueChecker,
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324181290
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324146082
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324146126
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324162028
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324161934
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
Rodney Aaron Stainback created SOLR-13758:
-
Summary: Ability to create custom metrics
Key: SOLR-13758
URL: https://issues.apache.org/jira/browse/SOLR-13758
Project: Solr
Issue Type:
atris commented on issue #876: Use The Passed In Threshold Value in
doConcurrentSearchWithThreshold
URL: https://github.com/apache/lucene-solr/pull/876#issuecomment-531226956
Can we merge this one? This is a simple fix to an existing test method.
jpountz merged pull request #868: LUCENE-8975: Code Cleanup: Use entryset for
map iteration wherever possible.
URL: https://github.com/apache/lucene-solr/pull/868
This is an automated message from the Apache Git Service.
To
atris commented on issue #815: LUCENE-8213: Introduce Asynchronous Caching in
LRUQueryCache
URL: https://github.com/apache/lucene-solr/pull/815#issuecomment-531221868
Any further thoughts on this one?
This is an automated
jpountz commented on issue #875: merge master
URL: https://github.com/apache/lucene-solr/pull/875#issuecomment-531221639
Probably open by mistake.
This is an automated message from the Apache Git Service.
To respond to the
jpountz closed pull request #875: merge master
URL: https://github.com/apache/lucene-solr/pull/875
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
jpountz commented on a change in pull request #865: LUCENE-8973: XYRectangle2D
should work on float space
URL: https://github.com/apache/lucene-solr/pull/865#discussion_r324172772
##
File path: lucene/sandbox/src/java/org/apache/lucene/geo/XYRectangle2D.java
##
@@ -16,42
iverase commented on issue #770: LUCENE-8746: Component2D topology library that
works on encoded space
URL: https://github.com/apache/lucene-solr/pull/770#issuecomment-531219042
Closing this PR. I am opening a new one with narrow scope
iverase closed pull request #770: LUCENE-8746: Component2D topology library
that works on encoded space
URL: https://github.com/apache/lucene-solr/pull/770
This is an automated message from the Apache Git Service.
To
[
https://issues.apache.org/jira/browse/LUCENE-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929154#comment-16929154
]
Adrien Grand commented on LUCENE-8970:
--
Maybe we can try to do a quick hack to see how much it
Build:
https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/79/
No tests ran.
Build Log:
[...truncated 25 lines...]
ERROR: Failed to check out http://svn.apache.org/repos/asf/lucene/test-data
org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the
server
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324145887
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929118#comment-16929118
]
Uwe Schindler commented on LUCENE-8951:
---
Not sure how we handle the "announce list" only. We had
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929116#comment-16929116
]
Uwe Schindler commented on LUCENE-8951:
---
Jan,
I will take care of ASF Jenkins after lunch. I am
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929113#comment-16929113
]
Jan Høydahl commented on LUCENE-8951:
-
Thanks Uwe. Will you try to redirect Apache Jenkins as well?
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324141173
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324141301
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
[
https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929102#comment-16929102
]
Uwe Schindler commented on LUCENE-8951:
---
Hi, INFRA-19013 looks like solved, but there were 2
KoenDG commented on a change in pull request #868: LUCENE-8975: Code Cleanup:
Use entryset for map iteration wherever possible.
URL: https://github.com/apache/lucene-solr/pull/868#discussion_r324133535
##
File path: lucene/CHANGES.txt
##
@@ -65,6 +65,8 @@ Other
*
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324130344
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324130518
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324125480
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324122789
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324130957
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324130644
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324129841
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324116784
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531174857
@jimczi Thanks for the comments -- updated the PR. Please let me know your
thoughts
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324105474
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324105474
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324103471
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
@@
atris commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324103214
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jpountz commented on a change in pull request #868: LUCENE-8975: Code Cleanup:
Use entryset for map iteration wherever possible.
URL: https://github.com/apache/lucene-solr/pull/868#discussion_r324101947
##
File path: lucene/CHANGES.txt
##
@@ -65,6 +65,8 @@ Other
*
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324098325
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
jimczi commented on a change in pull request #877: LUCENE-8978: Maximal Bottom
Score Based Early Termination
URL: https://github.com/apache/lucene-solr/pull/877#discussion_r324097168
##
File path:
lucene/core/src/java/org/apache/lucene/search/BottomValueChecker.java
##
atris commented on issue #877: LUCENE-8978: Maximal Bottom Score Based Early
Termination
URL: https://github.com/apache/lucene-solr/pull/877#issuecomment-531142523
I ran luceneutil with concurrent mode and wikimedium2m -- no degradation to
QPS and some tasks show a reduction of around 30%
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-8.x/209/
No tests ran.
Build Log:
[...truncated 14 lines...]
ERROR: Timeout after 10 minutes
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from
[
https://issues.apache.org/jira/browse/LUCENE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929036#comment-16929036
]
ASF subversion and git services commented on LUCENE-8966:
-
Commit
[
https://issues.apache.org/jira/browse/LUCENE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Ferenczi updated LUCENE-8966:
-
Fix Version/s: 8.3
master (9.0)
Resolution: Fixed
Status:
[
https://issues.apache.org/jira/browse/LUCENE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929035#comment-16929035
]
ASF subversion and git services commented on LUCENE-8966:
-
Commit
[
https://issues.apache.org/jira/browse/LUCENE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929032#comment-16929032
]
ASF subversion and git services commented on LUCENE-8966:
-
Commit
[
https://issues.apache.org/jira/browse/LUCENE-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929030#comment-16929030
]
Atri Sharma commented on LUCENE-8978:
-
||Task ('HighSpanNear', None)||P50 Base 11.060489||P50 Cmp
[
https://issues.apache.org/jira/browse/LUCENE-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929027#comment-16929027
]
Jim Ferenczi commented on LUCENE-8977:
--
I wonder why you think that this is an issue. Punctuations
90 matches
Mail list logo