[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1951 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1951/ No tests ran. Build Log: [...truncated 25 lines...] ERROR: Failed to check out http://svn.apache.org/repos/asf/lucene/test-data org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server svn: E175002: OPTIONS request failed on '/repos/asf/lucene/test-data' at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:112) at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:96) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:765) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:910) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:702) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:113) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1035) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:164) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.getRevisionNumber(SvnNgRepositoryAccess.java:119) at org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.getLocations(SvnRepositoryAccess.java:178) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepositoryFor(SvnNgRepositoryAccess.java:43) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.checkout(SvnNgAbstractUpdate.java:831) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:26) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:11) at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20) at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21) at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239) at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294) at hudson.scm.subversion.CheckoutUpdater$SubversionUpdateTask.perform(CheckoutUpdater.java:133) at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168) at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:176) at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:134) at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168) at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.tmatesoft.svn.core.internal.util.SVNSocketConnection.run(SVNSocketConnection.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ... 4 more java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
Alias Id condundrum
It seems that the real time get handler doesn't play nice with aliases. The current (and past) behavior seems to be that it only works for the first collection listed in the alias. This seems to be pretty clearly a bug, as one certainly would expect the /get executed against an alias to either refuse to work with aliases or work across all collections in the alias rather than silently working only on the first collection. However this has opened another can of worms after some discussion with Erick on slack. What's the expected behavior for this handler in the event that the same ID shows up in both collections? My first impulse was it should return both, and then I looked at /select to see what it did, and found that /select on an alias to collections that contain duplicate ids is not in a happy state either since it seems to randomly return one or the other document, but not both (probably based on the order in which the docs are returned from sub-requests which is not deterministic). So from a user perspective I can see arguments for either of two behaviors (in both cases) but no reason to like the current behaviors which are silently giving results that are hiding the situation and not returning all documents. Reasonable Behavior 1: Throw an error if a second document with the same ID is encountered. Reasonable Behavior 2: Return all documents including both (or more) documents that have colliding ID's. I can think of scenarios where either would be desirable, so I would think that we want to make the behavior choice something that can be selected by users. For this I see two possible points at which the user might express their preference: 1. At Configuration time with an Alias Property 2. At query time with a query parameter. This also implies a down side to routed aliases in that it's probably possible to index the same ID multiple times if it repeats less often than the collection creation interval for time routing or doesn't repeat within the same category (for category routed), but the responses to queries may then hide the duplicates in a non-deterministic fashion which is clearly bad. I am possibly ok with just documenting that aliases require the user to provide their own guarantees about ID uniqueness too... though part of me really wants to have a mode that detects this problem for the user somehow... (=2=id seems to work, but requires active checking?) In any case, the behavior with /get not returning docs in any but the first collection probably needs to be fixed. Thoughts? -Gus -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321052124
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
Review comment:
actually, I guess in the code as it stands the prior assigment will actually
allow us to return the prior work, but we haven't assigned the immutable copy
anywhere so the next time this is called the return value will be mutable
If the prior line is removed *then* we repeat the work unless this is changed
too...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321052124
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
Review comment:
actually, I guess in the code as it stands the prior assigment will actually
allow us to return the prior work, but we haven't assigned the immutable copy
anywhere so the next time this is called the return value will be mutable
If the prior line is removed *then* we repeat the work...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13727) V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" instead of using regex pattern
[ https://issues.apache.org/jira/browse/SOLR-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-13727: Fix Version/s: 8.3 master (9.0) Resolution: Fixed Status: Resolved (was: Patch Available) > V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" > instead of using regex pattern > > > Key: SOLR-13727 > URL: https://issues.apache.org/jira/browse/SOLR-13727 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, v2 API >Affects Versions: 8.2 >Reporter: Megan Carey >Priority: Major > Labels: easyfix, patch > Fix For: master (9.0), 8.3 > > Attachments: SOLR-13727.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > When the HttpSolrClient is formatting a V2Request, it needs to change the > endpoint from the default "/solr/..." to "/api/...". It does so by simply > calling String.replace, which replaces the first instance of "/solr" in the > URL with "/api". > > In the case where the host's address starts with "solr" and the HTTP protocol > is appended, this call changes the address for the request. Example: > if baseUrl is "http://solr-host.com/8983/solr;, this call will change to > "http:/api-host.com:8983/solr" > > We should use a regex pattern to ensure that we're replacing the correct > portion of the URL. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321050856
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
}
}
}
-return this.apis;
+return syncedApis;
Review comment:
This probably didn't need too change. If the prev comment is fixed, this
will return the mutable version
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321050532
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
}
}
}
-return this.apis;
+return syncedApis;
Review comment:
And here we return the mutable collection???
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321050532
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
}
}
}
-return this.apis;
+return syncedApis;
Review comment:
And here we return the mutable collection???
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321050511
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -303,11 +306,11 @@ public String toString() {
}
});
- this.apis = ImmutableList.copyOf(apis);
+ syncedApis = ImmutableList.copyOf(syncedApis);
Review comment:
Shouldn't this be assigning to this.apis so that we don't have to do this
work and create a new list every time?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] yonik commented on issue #850: SOLR-13727: Bug fix for V2Request handling in HttpSolrClient
yonik commented on issue #850: SOLR-13727: Bug fix for V2Request handling in HttpSolrClient URL: https://github.com/apache/lucene-solr/pull/850#issuecomment-528167851 Committed manually to master & 8x. Thanks Megan! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] yonik closed pull request #850: SOLR-13727: Bug fix for V2Request handling in HttpSolrClient
yonik closed pull request #850: SOLR-13727: Bug fix for V2Request handling in HttpSolrClient URL: https://github.com/apache/lucene-solr/pull/850 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked locking bug is fixed.
gus-asf commented on a change in pull request #838: SOLR-13705 Double-checked
locking bug is fixed.
URL: https://github.com/apache/lucene-solr/pull/838#discussion_r321050422
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/SecurityConfHandler.java
##
@@ -260,22 +260,25 @@ public String toString() {
@Override
public Collection getApis() {
-if (apis == null) {
+Collection syncedApis = apis;
+if (syncedApis == null) {
synchronized (this) {
-if (apis == null) {
- Collection apis = new ArrayList<>();
+syncedApis = apis;
+if (syncedApis == null) {
+ syncedApis = new ArrayList<>();
+ apis = syncedApis;
Review comment:
Is this correct? Doesn't this line assigning syncedApis to apis expose the
partially configured empty list to the rest of the world as a non-immutable
collection while the rest of this method executes (which was the original
complaint)? I think without this line the patch is an improvement...(the
existing code has this problem too).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13727) V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" instead of using regex pattern
[ https://issues.apache.org/jira/browse/SOLR-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922993#comment-16922993 ] ASF subversion and git services commented on SOLR-13727: Commit c3a72475a6eb02ad953a4a3f1ad586fb41fbe24e in lucene-solr's branch refs/heads/branch_8x from Megan Carey [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c3a7247 ] SOLR-13727: Bug fix for V2Request handling in HttpSolrClient Using regex to validate baseUrl and replace path for V2Requests Changed to using Java.net.URL for validation + path replacement > V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" > instead of using regex pattern > > > Key: SOLR-13727 > URL: https://issues.apache.org/jira/browse/SOLR-13727 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, v2 API >Affects Versions: 8.2 >Reporter: Megan Carey >Priority: Major > Labels: easyfix, patch > Attachments: SOLR-13727.patch > > Time Spent: 1h > Remaining Estimate: 0h > > When the HttpSolrClient is formatting a V2Request, it needs to change the > endpoint from the default "/solr/..." to "/api/...". It does so by simply > calling String.replace, which replaces the first instance of "/solr" in the > URL with "/api". > > In the case where the host's address starts with "solr" and the HTTP protocol > is appended, this call changes the address for the request. Example: > if baseUrl is "http://solr-host.com/8983/solr;, this call will change to > "http:/api-host.com:8983/solr" > > We should use a regex pattern to ensure that we're replacing the correct > portion of the URL. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13655) Cut Over Collections.unmodifiableSet usages to Set.*
[
https://issues.apache.org/jira/browse/SOLR-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922987#comment-16922987
]
Erick Erickson commented on SOLR-13655:
---
{color:#00}Yeah, it’s there. For certain idioms, running “precommit” will
catch them, specifically “var” declarations and a particular form of diamond
operator. It doesn’t check for much else, so this won’t fail precommit.{color}
And the regex that checks for “var” misses some usages, one I saw recently was
try (var blah blah blah) {
}
It’s not a hard rule though. The problem is it complicates back-porting 9.0 ->
8.x and the general consensus was that we should refrain from stuff that
complicated backports. See: SOLR-13658
And why the heck, when I follow a commit sha and click on the "diff" link for
any files that changed does it bive me a 403 "You don't have permission to
access this resource." error? Annoying.
> Cut Over Collections.unmodifiableSet usages to Set.*
>
>
> Key: SOLR-13655
> URL: https://issues.apache.org/jira/browse/SOLR-13655
> Project: Solr
> Issue Type: Improvement
>Reporter: Atri Sharma
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13727) V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" instead of using regex pattern
[ https://issues.apache.org/jira/browse/SOLR-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922988#comment-16922988 ] ASF subversion and git services commented on SOLR-13727: Commit 8c796b5f463c9c2529f29f317aa2c847f390a174 in lucene-solr's branch refs/heads/master from Megan Carey [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8c796b5 ] SOLR-13727: Bug fix for V2Request handling in HttpSolrClient Using regex to validate baseUrl and replace path for V2Requests Changed to using Java.net.URL for validation + path replacement > V2Requests: HttpSolrClient replaces first instance of "/solr" with "/api" > instead of using regex pattern > > > Key: SOLR-13727 > URL: https://issues.apache.org/jira/browse/SOLR-13727 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, v2 API >Affects Versions: 8.2 >Reporter: Megan Carey >Priority: Major > Labels: easyfix, patch > Attachments: SOLR-13727.patch > > Time Spent: 1h > Remaining Estimate: 0h > > When the HttpSolrClient is formatting a V2Request, it needs to change the > endpoint from the default "/solr/..." to "/api/...". It does so by simply > calling String.replace, which replaces the first instance of "/solr" in the > URL with "/api". > > In the case where the host's address starts with "solr" and the HTTP protocol > is appended, this call changes the address for the request. Example: > if baseUrl is "http://solr-host.com/8983/solr;, this call will change to > "http:/api-host.com:8983/solr" > > We should use a regex pattern to ensure that we're replacing the correct > portion of the URL. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] yonik commented on a change in pull request #850: SOLR-13727: Bug fix for V2Request handling in HttpSolrClient
yonik commented on a change in pull request #850: SOLR-13727: Bug fix for
V2Request handling in HttpSolrClient
URL: https://github.com/apache/lucene-solr/pull/850#discussion_r321048686
##
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrClient.java
##
@@ -330,6 +333,12 @@ protected ModifiableSolrParams
calculateQueryParams(Set queryParamNames,
}
return queryModParams;
}
+
+ private String changeV2RequestEndpoint(String basePath) throws
MalformedURLException {
Review comment:
This is a widely used client class, so we should probably be slightly more
wary of adding public methods. Perhaps package protected static... until we
need it to be public for some reason. I'll make that change when I commit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
[
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-13741:
Attachment: SOLR-13741.patch
Status: Open (was: Open)
Attaching my patch, note that at the moment this patch only modifies
{{AuditLoggerIntegrationTest}} and does not yet address the '#1' comment I made
above regarding the 'delay' option on {{CallbackAuditLoggerPlugin}} – there are
additional nocommit comments regarding the planned changes for that, but I
didn't want to start on those changes until these existing uncertainties were
addressed.
[~janhoy] : I would greatly appreciate your review here to help clear up the
"correct test, bad behavior" vs "correct behavior, bad test" questions.
> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Hoss Man
>Assignee: Hoss Man
>Priority: Major
> Attachments: SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from
> AuditLoggerIntegrationTest. When i started reading through that code, 2
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about
> CPU scheduling that aren't neccessarily going to be true (and also suffers
> from the problem that Thread.sleep isn't garunteed to sleep as long as you
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll
> loop, until it contains at least N items -- but the code that adds items to
> that buffer in the async Callback thread async _before_ the code that updates
> other state variables (like the global {{count}} and the patch specific
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3
> events added to the buffer, but calling {{assertEquals(3,
> receiver.getTotalCount())}} could subsequently fail because that variable
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for
> that specific problem would be to update all other state _before_ adding the
> event to the buffer, I set out to try and make more general improvements to
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data
> structures
> * harden the assertions made about the expected events recieved (updating
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies
> between things the test currently claims/asserts, and what *actually* happens
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes
> copious nocommits about things seem suspect. the summary of these concerns
> is:
> * SolrException status codes that do not match what the existing test says
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected
> AuditEvent in the existing test -- which the current test seems to miss in
> it's assertions because it's picking up some extra events from triggered by
> previuos requests earlier in the test that just happen to also match the
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code
> bugs" or if the test is faulty.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
Hoss Man created SOLR-13741:
---
Summary: possible AuditLogger bugs uncovered while hardening
AuditLoggerIntegrationTest
Key: SOLR-13741
URL: https://issues.apache.org/jira/browse/SOLR-13741
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Hoss Man
Assignee: Hoss Man
A while back i saw a weird non-reproducible failure from
AuditLoggerIntegrationTest. When i started reading through that code, 2 things
jumped out at me:
# the way the 'delay' option works is brittle, and makes assumptions about CPU
scheduling that aren't neccessarily going to be true (and also suffers from the
problem that Thread.sleep isn't garunteed to sleep as long as you ask it too)
# the way the existing {{waitForAuditEventCallbacks(number)}} logic works by
checking the size of a (List) {{buffer}} of recieved events in a sleep/poll
loop, until it contains at least N items -- but the code that adds items to
that buffer in the async Callback thread async _before_ the code that updates
other state variables (like the global {{count}} and the patch specific
{{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3
events added to the buffer, but calling {{assertEquals(3,
receiver.getTotalCount())}} could subsequently fail because that variable
hadn't been udpated yet.
#2 was the source of the failures I was seeing, and while a quick fix for that
specific problem would be to update all other state _before_ adding the event
to the buffer, I set out to try and make more general improvements to the test:
* eliminate the dependency on sleep loops by {{await}}-ing on concurrent data
structures
* harden the assertions made about the expected events recieved (updating some
test methods that currently just assert the number of events recieved)
* add new assertions that _only_ the expected events are recieved.
In the process of doing this, I've found several oddities/descrepencies between
things the test currently claims/asserts, and what *actually* happens under
more rigerous scrutiny/assertions.
I'll attach a patch shortly that has my (in progress) updates and inlcudes
copious nocommits about things seem suspect. the summary of these concerns is:
* SolrException status codes that do not match what the existing test says they
should (but doesn't assert)
* extra AuditEvents occuring that the existing test does not expect
* AuditEvents for incorrect credentials that do not at all match the expected
AuditEvent in the existing test -- which the current test seems to miss in it's
assertions because it's picking up some extra events from triggered by previuos
requests earlier in the test that just happen to also match the asserctions.
...it's not clear to me if the test logic is correct and these are "code bugs"
or if the test is faulty.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13655) Cut Over Collections.unmodifiableSet usages to Set.*
[ https://issues.apache.org/jira/browse/SOLR-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922971#comment-16922971 ] Gus Heck commented on SOLR-13655: - I could have sworn there was a discussion that lead to the conclusion that Java 9+ idioms should wait until the first 9.0 to avoid complicating merges but I cant' for the life of me find it by searching in gmail... > Cut Over Collections.unmodifiableSet usages to Set.* > > > Key: SOLR-13655 > URL: https://issues.apache.org/jira/browse/SOLR-13655 > Project: Solr > Issue Type: Improvement >Reporter: Atri Sharma >Priority: Major > Fix For: master (9.0) > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #856: LUCENE-8965 SometimesConcurrentMergeScheduler
gus-asf commented on a change in pull request #856: LUCENE-8965
SometimesConcurrentMergeScheduler
URL: https://github.com/apache/lucene-solr/pull/856#discussion_r321035131
##
File path:
lucene/core/src/java/org/apache/lucene/index/SometimesConcurrentMergeScheduler.java
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.index;
+
+import java.io.IOException;
+import java.util.Observable;
+import java.util.Observer;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.lucene.util.ThreadInterruptedException;
+
+/**
+ * A variant of CMS: If there are cheap merges, wait for a merge to complete
before continuing. This
+ * has the benefit of greatly increasing the odds that an {@link
org.apache.lucene.search.IndexSearcher}
+ * will see fewer segments. Normally, CMS does all merges concurrently, and so
it won't be until the
+ * next commit that the IndexSearcher benefits from fewer segments. The
trade-off is less
+ * concurrency, and there will be some delay on a segment flush for a cheap
merge if present.
+ *
+ * @author dsmiley
+ * @since solr.7
+ */
+public class SometimesConcurrentMergeScheduler extends
ConcurrentMergeScheduler {
+
+ private long cheapMergeThresholdBytes = 2 * 1024 * 1024; // 2MB
+ private long cheapMaxMergeWaitTimeoutMs = TimeUnit.SECONDS.toMillis(2);
Review comment:
oops see setter below :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #856: LUCENE-8965 SometimesConcurrentMergeScheduler
gus-asf commented on a change in pull request #856: LUCENE-8965
SometimesConcurrentMergeScheduler
URL: https://github.com/apache/lucene-solr/pull/856#discussion_r321034932
##
File path:
lucene/core/src/java/org/apache/lucene/index/SometimesConcurrentMergeScheduler.java
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.index;
+
+import java.io.IOException;
+import java.util.Observable;
+import java.util.Observer;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.lucene.util.ThreadInterruptedException;
+
+/**
+ * A variant of CMS: If there are cheap merges, wait for a merge to complete
before continuing. This
+ * has the benefit of greatly increasing the odds that an {@link
org.apache.lucene.search.IndexSearcher}
+ * will see fewer segments. Normally, CMS does all merges concurrently, and so
it won't be until the
+ * next commit that the IndexSearcher benefits from fewer segments. The
trade-off is less
+ * concurrency, and there will be some delay on a segment flush for a cheap
merge if present.
+ *
+ * @author dsmiley
+ * @since solr.7
+ */
+public class SometimesConcurrentMergeScheduler extends
ConcurrentMergeScheduler {
+
+ private long cheapMergeThresholdBytes = 2 * 1024 * 1024; // 2MB
+ private long cheapMaxMergeWaitTimeoutMs = TimeUnit.SECONDS.toMillis(2);
Review comment:
Hard coded timeout...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on issue #775: LUCENE-8910: upgrade to icu 62.1 must be completed
rmuir commented on issue #775: LUCENE-8910: upgrade to icu 62.1 must be completed URL: https://github.com/apache/lucene-solr/pull/775#issuecomment-528146605 looks good to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922953#comment-16922953
]
ASF subversion and git services commented on SOLR-13105:
Commit 42ab37d3d2bbfc528cc1a7fadc7a6a6b51aaac77 in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=42ab37d ]
SOLR-13105: Change cover title
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8956) QueryRescorer sort optimization
[
https://issues.apache.org/jira/browse/LUCENE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922944#comment-16922944
]
Lucene/Solr QA commented on LUCENE-8956:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 16m
2s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8956 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12979474/LUCENE-8956.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 5204d0f963c |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/204/testReport/ |
| modules | C: lucene/core U: lucene/core |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/204/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> QueryRescorer sort optimization
> ---
>
> Key: LUCENE-8956
> URL: https://issues.apache.org/jira/browse/LUCENE-8956
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/query/scoring
>Reporter: Paul Sanwald
>Priority: Minor
> Attachments: LUCENE-8956.patch
>
>
> This patch addresses a TODO in QueryRescorer: We should not sort the full
> array of the results returned from rescoring, but rather only topN, when topN
> is less than total hits.
>
> Made this optimization with some suggestions from [~jpountz] and [~jimczi],
> this is my first lucene patch submission.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #823: LUCENE-8939: Introduce Shared Count Early Termination In Parallel Search
mikemccand commented on a change in pull request #823: LUCENE-8939: Introduce
Shared Count Early Termination In Parallel Search
URL: https://github.com/apache/lucene-solr/pull/823#discussion_r320956707
##
File path:
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java
##
@@ -191,32 +195,64 @@ public static TopScoreDocCollector create(int numHits,
int totalHitsThreshold) {
* objects.
*/
public static TopScoreDocCollector create(int numHits, ScoreDoc after, int
totalHitsThreshold) {
+return create(numHits, after,
HitsThresholdChecker.create(totalHitsThreshold));
+ }
+
+ static TopScoreDocCollector create(int numHits, ScoreDoc after,
HitsThresholdChecker hitsThresholdChecker) {
if (numHits <= 0) {
throw new IllegalArgumentException("numHits must be > 0; please use
TotalHitCountCollector if you just need the total hit count");
}
-if (totalHitsThreshold < 0) {
- throw new IllegalArgumentException("totalHitsThreshold must be >= 0, got
" + totalHitsThreshold);
+if (hitsThresholdChecker == null) {
+ throw new IllegalArgumentException("hitsThresholdChecker must be non
null");
}
if (after == null) {
- return new SimpleTopScoreDocCollector(numHits, totalHitsThreshold);
+ return new SimpleTopScoreDocCollector(numHits, hitsThresholdChecker);
} else {
- return new PagingTopScoreDocCollector(numHits, after,
totalHitsThreshold);
+ return new PagingTopScoreDocCollector(numHits, after,
hitsThresholdChecker);
}
}
- final int totalHitsThreshold;
+ /**
+ * Create a CollectorManager which uses a shared hit counter to maintain
number of hits
+ */
+ public static CollectorManager
createSharedManager(int numHits, FieldDoc after,
+
int totalHitsThreshold) {
+return new CollectorManager<>() {
+
+ private final HitsThresholdChecker hitsThresholdChecker =
HitsThresholdChecker.createShared(totalHitsThreshold);
Review comment:
Same here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #823: LUCENE-8939: Introduce Shared Count Early Termination In Parallel Search
mikemccand commented on a change in pull request #823: LUCENE-8939: Introduce
Shared Count Early Termination In Parallel Search
URL: https://github.com/apache/lucene-solr/pull/823#discussion_r320956589
##
File path: lucene/core/src/java/org/apache/lucene/search/TopFieldCollector.java
##
@@ -410,10 +423,35 @@ public static TopFieldCollector create(Sort sort, int
numHits, FieldDoc after,
throw new IllegalArgumentException("after.fields has " +
after.fields.length + " values but sort has " + sort.getSort().length);
}
- return new PagingFieldCollector(sort, queue, after, numHits,
totalHitsThreshold);
+ return new PagingFieldCollector(sort, queue, after, numHits,
hitsThresholdChecker);
}
}
+ /**
+ * Create a CollectorManager which uses a shared hit counter to maintain
number of hits
+ */
+ public static CollectorManager
createSharedManager(Sort sort, int numHits, FieldDoc after,
+
int totalHitsThreshold) {
+return new CollectorManager<>() {
+
+ private final HitsThresholdChecker hitsThresholdChecker =
HitsThresholdChecker.createShared(totalHitsThreshold);
Review comment:
Maybe newline before the `@Override` here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13240) UTILIZENODE action results in an exception
[
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922828#comment-16922828
]
Lucene/Solr QA commented on SOLR-13240:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
49s{color} | {color:green} solrj in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13240 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12979415/SOLR-13240.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 5204d0f |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/548/testReport/ |
| modules | C: solr/solrj U: solr/solrj |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/548/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Assignee: Christine Poerschke
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it
> moved a few replicas:
> {
> "responseHeader":{
> "status":500,
> "QTime":40220},
> "Operation utilizenode caused
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
> Comparison method violates its general contract!",
> "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates
> its general contract!\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>
>
[jira] [Updated] (LUCENE-8884) Add Directory wrapper to track per-query IO counters
[
https://issues.apache.org/jira/browse/LUCENE-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-8884:
---
Attachment: LUCENE-8884.patch
Status: Open (was: Open)
Another iteration folding [~rcmuir]'s feedback.
I was worried that the thread that called {{clone()}} may not be the same
thread that then consumes the {{IndexInput}} and added an assertion, but it
looks like it's OK. I added another random test too, and improved javadocs.
I could not eliminate the required {{setKeyForThread}} call because even in the
single threaded case, where only one thread executes the query across all
segments, the directory wrapper still needs to know which query that is to
track its IO counters.
I haven't tested performance impact of this but it's likely minor now since we
now retrieve the counters on {{clone()}} instead of on every IO operation.
> Add Directory wrapper to track per-query IO counters
>
>
> Key: LUCENE-8884
> URL: https://issues.apache.org/jira/browse/LUCENE-8884
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/store
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>Priority: Minor
> Attachments: LUCENE-8884.patch, LUCENE-8884.patch
>
>
> Lucene's IO abstractions ({{Directory, IndexInput/Output}}) make it really
> easy to track counters of how many IOPs and net bytes are read for each
> query, which is a useful metric to track/aggregate/alarm on in production or
> dev benchmarks.
> At my day job we use these wrappers in our nightly benchmarks to catch any
> accidental performance regressions.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 466 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/466/
All tests passed
Build Log:
[...truncated 64019 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj1276441538
[ecj-lint] Compiling 1289 source files to /tmp/ecj1276441538
[ecj-lint] Processing annotations
[ecj-lint] Annotations processed
[ecj-lint] Processing annotations
[ecj-lint] No elements to process
[ecj-lint] invalid Class-Path header in manifest of jar file:
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file:
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] --
[ecj-lint] 1. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java
(at line 219)
[ecj-lint] return (NamedList) new
JavaBinCodec(resolver).unmarshal(in);
[ecj-lint]^^
[ecj-lint] Resource leak: '' is never closed
[ecj-lint] --
[ecj-lint] --
[ecj-lint] 2. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java
(at line 232)
[ecj-lint] ReplicationHandler replicationHandler = (ReplicationHandler)
handler;
[ecj-lint]^^
[ecj-lint] Resource leak: 'replicationHandler' is never closed
[ecj-lint] --
[ecj-lint] 3. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java
(at line 628)
[ecj-lint] PeerSyncWithLeader peerSyncWithLeader = new
PeerSyncWithLeader(core,
[ecj-lint]^^
[ecj-lint] Resource leak: 'peerSyncWithLeader' is never closed
[ecj-lint] --
[ecj-lint] --
[ecj-lint] 4. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/cloud/SyncStrategy.java
(at line 186)
[ecj-lint] PeerSync peerSync = new PeerSync(core, syncWith,
core.getUpdateHandler().getUpdateLog().getNumRecordsToKeep(), true,
peerSyncOnlyWithActive, false);
[ecj-lint]
[ecj-lint] Resource leak: 'peerSync' is never closed
[ecj-lint] --
[ecj-lint] --
[ecj-lint] 5. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
(at line 788)
[ecj-lint] throw new UnsupportedOperationException("must add at least 1
node first");
[ecj-lint]
^^
[ecj-lint] Resource leak: 'queryRequest' is not closed at this location
[ecj-lint] --
[ecj-lint] 6. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
(at line 794)
[ecj-lint] throw new UnsupportedOperationException("must add at least 1
node first");
[ecj-lint]
^^
[ecj-lint] Resource leak: 'queryRequest' is not closed at this location
[ecj-lint] --
[ecj-lint] --
[ecj-lint] 7. WARNING in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/core/CoreContainer.java
(at line 726)
[ecj-lint] SolrFieldCacheBean fieldCacheBean = new SolrFieldCacheBean();
[ecj-lint]^^
[ecj-lint] Resource leak: 'fieldCacheBean' is never closed
[ecj-lint] --
[ecj-lint] --
[ecj-lint] 8. ERROR in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
(at line 19)
[ecj-lint] import javax.naming.Context;
[ecj-lint]
[ecj-lint] The type javax.naming.Context is not accessible
[ecj-lint] --
[ecj-lint] 9. ERROR in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
(at line 20)
[ecj-lint] import javax.naming.InitialContext;
[ecj-lint]^^^
[ecj-lint] The type javax.naming.InitialContext is not accessible
[ecj-lint] --
[ecj-lint] 10. ERROR in
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
(at line 21)
[ecj-lint] import javax.naming.NamingException;
[ecj-lint]
[ecj-lint] The type javax.naming.NamingException is not accessible
[ecj-lint] --
[ecj-lint] 11. ERROR in
[jira] [Updated] (SOLR-13738) UnifiedHighlighter
[
https://issues.apache.org/jira/browse/SOLR-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-13738:
Summary: UnifiedHighlighter (was: RequestHandlerBase ...
ClassCastException: class ... .lucene.search.IndexSearcher cannot be cast to
class ... .solr.search.SolrIndexSearcher ...)
> UnifiedHighlighter
> --
>
> Key: SOLR-13738
> URL: https://issues.apache.org/jira/browse/SOLR-13738
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.2
>Reporter: Jochen Barth
>Priority: Major
>
> Mikhail Khludnev said, this is a bug;
> Here the complete error message for the Query below:
> Just tested wirth 8.1.1: works.
> {quote}
> 2019-08-30 12:40:40.476 ERROR (qtp2116511124-65) [ x:Suchindex]
> o.a.s.h.RequestHandlerBase java.lang.ClassCastException: class
> org.apache.lucene.search.IndexSearcher cannot be cast to class
> org.apache.solr.search.SolrIndexSearcher (or
> g.apache.lucene.search.IndexSearcher and
> org.apache.solr.search.SolrIndexSearcher are in unnamed module of loader
> org.eclipse.jetty.webapp.WebAppClassLoader @5ed190be)
> at
> org.apache.solr.search.join.GraphQuery.createWeight(GraphQuery.java:115)
> at
> org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumsWeightMatcher(FieldOffsetStrategy.java:137)
> at
> org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumFromReader(FieldOffsetStrategy.java:74)
> at
> org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.getOffsetsEnum(MemoryIndexOffsetStrategy.java:110)
> at
> org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:76)
> at
> org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:641)
> at
> org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:510)
> at
> org.apache.solr.highlight.UnifiedSolrHighlighter.doHighlighting(UnifiedSolrHighlighter.java:149)
> at
> org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:171)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
[jira] [Updated] (SOLR-13738) UnifiedHighlighter can't highlight GraphQuery
[
https://issues.apache.org/jira/browse/SOLR-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-13738:
Summary: UnifiedHighlighter can't highlight GraphQuery (was:
UnifiedHighlighter)
> UnifiedHighlighter can't highlight GraphQuery
> -
>
> Key: SOLR-13738
> URL: https://issues.apache.org/jira/browse/SOLR-13738
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.2
>Reporter: Jochen Barth
>Priority: Major
>
> Mikhail Khludnev said, this is a bug;
> Here the complete error message for the Query below:
> Just tested wirth 8.1.1: works.
> {quote}
> 2019-08-30 12:40:40.476 ERROR (qtp2116511124-65) [ x:Suchindex]
> o.a.s.h.RequestHandlerBase java.lang.ClassCastException: class
> org.apache.lucene.search.IndexSearcher cannot be cast to class
> org.apache.solr.search.SolrIndexSearcher (or
> g.apache.lucene.search.IndexSearcher and
> org.apache.solr.search.SolrIndexSearcher are in unnamed module of loader
> org.eclipse.jetty.webapp.WebAppClassLoader @5ed190be)
> at
> org.apache.solr.search.join.GraphQuery.createWeight(GraphQuery.java:115)
> at
> org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumsWeightMatcher(FieldOffsetStrategy.java:137)
> at
> org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumFromReader(FieldOffsetStrategy.java:74)
> at
> org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.getOffsetsEnum(MemoryIndexOffsetStrategy.java:110)
> at
> org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:76)
> at
> org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:641)
> at
> org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:510)
> at
> org.apache.solr.highlight.UnifiedSolrHighlighter.doHighlighting(UnifiedSolrHighlighter.java:149)
> at
> org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:171)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:505)
>
[jira] [Created] (SOLR-13740) Assert returning ExtendedFileField
Mikhail Khludnev created SOLR-13740:
---
Summary: Assert returning ExtendedFileField
Key: SOLR-13740
URL: https://issues.apache.org/jira/browse/SOLR-13740
Project: Solr
Issue Type: Test
Security Level: Public (Default Security Level. Issues are Public)
Components: Schema and Analysis
Reporter: Mikhail Khludnev
It works, commit sometimes later
{code}
diff --git
a/solr/core/src/test/org/apache/solr/schema/ExternalFileFieldSortTest.java
b/solr/core/src/test/org/apache/solr/schema/ExternalFileFieldSortTest.java
index 632b413..4106e15 100644
--- a/solr/core/src/test/org/apache/solr/schema/ExternalFileFieldSortTest.java
+++ b/solr/core/src/test/org/apache/solr/schema/ExternalFileFieldSortTest.java
@@ -48,8 +48,9 @@
addDocuments();
assertQ("query",
-req("q", "*:*", "sort", "eff asc"),
+req("q", "*:*", "sort", "eff asc", "fl", "id,field(eff)"),
"//result/doc[position()=1]/str[.='3']",
+"//result/doc[position()=1]/float[@name='field(eff)' and .='0.001']",
"//result/doc[position()=2]/str[.='1']",
"//result/doc[position()=10]/str[.='8']");
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922794#comment-16922794
]
ASF subversion and git services commented on SOLR-13105:
Commit b64c8cc5eaf9bd7e940a72038863385368ab011f in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b64c8cc ]
SOLR-13105: Change cover image
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley opened a new pull request #856: LUCENE-8965 SometimesConcurrentMergeScheduler
dsmiley opened a new pull request #856: LUCENE-8965 SometimesConcurrentMergeScheduler URL: https://github.com/apache/lucene-solr/pull/856 The initial version here is copied with simple modifications from one I wrote at Salesforce. I do not propose we actually commit this version, it's more of a straw-man and show & tell. Instead of the awkward subclassing, it'd be nicer to enhance CMS directly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13655) Cut Over Collections.unmodifiableSet usages to Set.*
[ https://issues.apache.org/jira/browse/SOLR-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922761#comment-16922761 ] Tomás Fernández Löbbe commented on SOLR-13655: -- Yes, fair point. I decided to go ahead with this change because the change is limited and simple(one line per occurrence), but I agree, these changes will make backporting more complicated. > Cut Over Collections.unmodifiableSet usages to Set.* > > > Key: SOLR-13655 > URL: https://issues.apache.org/jira/browse/SOLR-13655 > Project: Solr > Issue Type: Improvement >Reporter: Atri Sharma >Priority: Major > Fix For: master (9.0) > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jimczi commented on issue #854: Shared PQ Based Early Termination for Concurrent Search
jimczi commented on issue #854: Shared PQ Based Early Termination for Concurrent Search URL: https://github.com/apache/lucene-solr/pull/854#issuecomment-528035227 I thought that the follow up for LUCENE-8939 would be to allow the sharing of the minimum score (could be extended to a minimum FieldDoc) across slices ? Sharing the minimum score (or minimum FieldDoc) requires very little synchronization while a global priority queue seems much more costly. The other advantage is that we could add this ability in the current topdocs collector like we did for LUCENE-8939. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8965) ConcurrentMergeScheduler should maybe sometimes do synchronously
David Smiley created LUCENE-8965: Summary: ConcurrentMergeScheduler should maybe sometimes do synchronously Key: LUCENE-8965 URL: https://issues.apache.org/jira/browse/LUCENE-8965 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: David Smiley Assignee: David Smiley It can be beneficial for the ConcurrentMergeScheduler to sometimes _not_ do concurrent merges (i.e. sometimes do a serial merge). When the provided "OneMerge" is _small_ and when the MergeTrigger is FULL_FLUSH (i.e. on a commit), a new searcher would benefit from seeing fewer segments. If index replication is used, this setting can reduce the net replication by merging a little more eagerly sometimes. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8956) QueryRescorer sort optimization
[ https://issues.apache.org/jira/browse/LUCENE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922754#comment-16922754 ] Paul Sanwald commented on LUCENE-8956: -- [~jpountz] I've updated the patch with a new test :) > QueryRescorer sort optimization > --- > > Key: LUCENE-8956 > URL: https://issues.apache.org/jira/browse/LUCENE-8956 > Project: Lucene - Core > Issue Type: Improvement > Components: core/query/scoring >Reporter: Paul Sanwald >Priority: Minor > Attachments: LUCENE-8956.patch > > > This patch addresses a TODO in QueryRescorer: We should not sort the full > array of the results returned from rescoring, but rather only topN, when topN > is less than total hits. > > Made this optimization with some suggestions from [~jpountz] and [~jimczi], > this is my first lucene patch submission. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922753#comment-16922753
]
ASF subversion and git services commented on SOLR-13105:
Commit 149e85aa7f298817ab2b5cfa31d130ba65748975 in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=149e85a ]
SOLR-13105: Revamp dsp docs 1
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8956) QueryRescorer sort optimization
[ https://issues.apache.org/jira/browse/LUCENE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Sanwald updated LUCENE-8956: - Attachment: LUCENE-8956.patch > QueryRescorer sort optimization > --- > > Key: LUCENE-8956 > URL: https://issues.apache.org/jira/browse/LUCENE-8956 > Project: Lucene - Core > Issue Type: Improvement > Components: core/query/scoring >Reporter: Paul Sanwald >Priority: Minor > Attachments: LUCENE-8956.patch > > > This patch addresses a TODO in QueryRescorer: We should not sort the full > array of the results returned from rescoring, but rather only topN, when topN > is less than total hits. > > Made this optimization with some suggestions from [~jpountz] and [~jimczi], > this is my first lucene patch submission. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8956) QueryRescorer sort optimization
[ https://issues.apache.org/jira/browse/LUCENE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Sanwald updated LUCENE-8956: - Attachment: (was: LUCENE-8956.patch) > QueryRescorer sort optimization > --- > > Key: LUCENE-8956 > URL: https://issues.apache.org/jira/browse/LUCENE-8956 > Project: Lucene - Core > Issue Type: Improvement > Components: core/query/scoring >Reporter: Paul Sanwald >Priority: Minor > Attachments: LUCENE-8956.patch > > > This patch addresses a TODO in QueryRescorer: We should not sort the full > array of the results returned from rescoring, but rather only topN, when topN > is less than total hits. > > Made this optimization with some suggestions from [~jpountz] and [~jimczi], > this is my first lucene patch submission. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922749#comment-16922749
]
David Smiley commented on LUCENE-8962:
--
The existing MergeTrigger mechanism is a way to conditionally activate this
stuff. We already did that; I forgot to mention it. Thus it's only on a
commit (not a flush) that activates this behavior. I'll create an issue for
the merge scheduler and link here.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922731#comment-16922731
]
ASF subversion and git services commented on SOLR-13105:
Commit 892a005afff30e81b1e691123f02433fb6b9af5f in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=892a005 ]
SOLR-13105: Revamp dsp docs
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8964) Allow GeoJSON parser to properly skip string arrays
Alexander Reelsen created LUCENE-8964: - Summary: Allow GeoJSON parser to properly skip string arrays Key: LUCENE-8964 URL: https://issues.apache.org/jira/browse/LUCENE-8964 Project: Lucene - Core Issue Type: Improvement Components: core/other Affects Versions: trunk Reporter: Alexander Reelsen Attachments: lucene-parse-geojson-arrays-0.patch The Geo JSON parser throws an exception when trying to parse an array of strings, which is somewhat common in some free geojson services like [https://whosonfirst.org|https://whosonfirst.org/] An example file can be seen at [https://data.whosonfirst.org/101/748/479/101748479.geojson] This fixes the parser to also parse a string array. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1442 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1442/ No tests ran. Build Log: [...truncated 24456 lines...] [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [java] Processed 2595 links (2121 relative) to 3660 anchors in 260 files [echo] Validated Links & Anchors via: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/ -dist-changes: [copy] Copying 4 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes package: -unpack-solr-tgz: -ensure-solr-tgz-exists: [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked [untar] Expanding: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-9.0.0.tgz into /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked generate-maven-artifacts: resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922645#comment-16922645
]
ASF subversion and git services commented on SOLR-13105:
Commit 0f085fd1b56c0d0b666a320f04f57594cf8f98bb in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0f085fd ]
SOLR-13105: Revamp simulations docs 17
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on issue #828: LUCENE-8753: UniformSplitPostingsFormat
dsmiley commented on issue #828: LUCENE-8753: UniformSplitPostingsFormat URL: https://github.com/apache/lucene-solr/pull/828#issuecomment-527974721 Thanks; looks good Bruno. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13240) UTILIZENODE action results in an exception
[
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922627#comment-16922627
]
Christine Poerschke commented on SOLR-13240:
Attached patch rebases against the latest master which includes the above
mentioned SOLR-13736 TestPolicy.testNodeLostMultipleReplica refactor.
{quote}... My interpretion of the revised test expectations is that now then
the "a node can have 2 shardX replicas" condition is no longer being met ...
further adjustments to that sub-test might be needed to preserve the intended
coverage. ...
{quote}
The now reduced patch complexity makes it easier to reason about how the test's
"expected values" did and did not change:
* node1 is the lost node and thus r1, r3 and r5 need to be moved off it and
that continues to happen, only the order of the moves has changed.
* Previously the three moves were to node2 (two moves) and node3 (one move)
and now the moves are to node2 (one move) and node3 (two moves).
* In the list of replicas being moved we have for shard2 the r3 and r5
replicas:
** Initially it seems them now moving to node3 and node2 respectively means
that the "a node can have 2 shard2 replicas" intention of the test is no longer
being met.
** However looking closer at
[testPolicy.json|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.2.0/solr/solrj/src/test-files/solrj/solr/autoscaling/testPolicy.json]
(and the test's previous moves off node1) it becomes clear that r6 lives on
node3 and thus r3 joining r6 on node3 would still result in two shard2 replicas
on the same (node3) node i.e. the "a node can have 2 shard2 replicas" intention
of the test continues to be met.
Assuming 'green' feedback from Lucene/Solr QA and if there are no further
comments or concern here then I'll aim to commit this ticket sometime tomorrow
afternoon (Thursday) or Friday.
> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it
> moved a few replicas:
> {
> "responseHeader":{
> "status":500,
> "QTime":40220},
> "Operation utilizenode caused
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
> Comparison method violates its general contract!",
> "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates
> its general contract!\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)\n\tat
>
>
[jira] [Assigned] (SOLR-13240) UTILIZENODE action results in an exception
[
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke reassigned SOLR-13240:
--
Assignee: Christine Poerschke
> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Assignee: Christine Poerschke
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it
> moved a few replicas:
> {
> "responseHeader":{
> "status":500,
> "QTime":40220},
> "Operation utilizenode caused
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
> Comparison method violates its general contract!",
> "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates
> its general contract!\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:531)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat
>
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
>
>
[jira] [Updated] (SOLR-13240) UTILIZENODE action results in an exception
[
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-13240:
---
Attachment: SOLR-13240.patch
> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch,
> SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it
> moved a few replicas:
> {
> "responseHeader":{
> "status":500,
> "QTime":40220},
> "Operation utilizenode caused
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
> Comparison method violates its general contract!",
> "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates
> its general contract!\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:531)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat
>
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat
>
>
[jira] [Updated] (SOLR-13736) Reduce code duplication in TestPolicy.testNodeLostMultipleReplica
[ https://issues.apache.org/jira/browse/SOLR-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-13736: --- Fix Version/s: 8.3 master (9.0) Resolution: Fixed Status: Resolved (was: Patch Available) > Reduce code duplication in TestPolicy.testNodeLostMultipleReplica > - > > Key: SOLR-13736 > URL: https://issues.apache.org/jira/browse/SOLR-13736 > Project: Solr > Issue Type: Test >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Fix For: master (9.0), 8.3 > > Attachments: SOLR-13736.patch > > > Splitting this refactor out from the SOLR-13240 changes in which it is > currently included. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13739) Managed resource observers have to be added only once
[ https://issues.apache.org/jira/browse/SOLR-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Wöckinger updated SOLR-13739: Affects Version/s: master (9.0) > Managed resource observers have to be added only once > - > > Key: SOLR-13739 > URL: https://issues.apache.org/jira/browse/SOLR-13739 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Server >Affects Versions: 8.0, 8.1, master (9.0), 8.2, 8.1.1 >Reporter: Thomas Wöckinger >Priority: Major > Labels: easyfix, performance > Time Spent: 10m > Remaining Estimate: 0h > > On huge schema modifications, mostly happen due to creation of a new > collection, the same observer instance of an ResourceLoaderAware component is > added again and again. > This leads to a runtime behaviour of n²/2 where n is is the number of schema > operation multiplied by ResourceLoaderAware components instead of the number > of containing ResourceLoaderAware components. > E.g. If you have 1000 schema operations and 2 ResourceLoaderAware components > this leads to 50 operations instead of 2000. > Even worse the corresponding resource is registered again and again, which > can take some time e.g. ManagedSynonymGraphFilterFactory needs about 5s on > each call (depending on the size of synonyms). -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant closed pull request #676: LUCENE-8753: SharedTerms UniformSplit
bruno-roustant closed pull request #676: LUCENE-8753: SharedTerms UniformSplit URL: https://github.com/apache/lucene-solr/pull/676 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant commented on issue #676: LUCENE-8753: SharedTerms UniformSplit
bruno-roustant commented on issue #676: LUCENE-8753: SharedTerms UniformSplit URL: https://github.com/apache/lucene-solr/pull/676#issuecomment-527962140 Moved to #828 so I am closing this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922609#comment-16922609 ] Bruno Roustant commented on LUCENE-8753: Ok, I followed your advice to include the "shared terms" extension (subpackage) in the same PR #828. I'm going to close the two previous ones. > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Assignee: David Smiley >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > Time Spent: 4h 20m > Remaining Estimate: 0h > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13733) add more class-level javadocs for public org.apache.solr.metrics classes
[
https://issues.apache.org/jira/browse/SOLR-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922604#comment-16922604
]
ASF subversion and git services commented on SOLR-13733:
Commit a8c0b9a6fab939934485cb157fcc6efd73f74249 in lucene-solr's branch
refs/heads/branch_8x from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a8c0b9a ]
SOLR-13733: add class-level javadocs for 4 org.apache.solr.metrics classes
> add more class-level javadocs for public org.apache.solr.metrics classes
>
>
> Key: SOLR-13733
> URL: https://issues.apache.org/jira/browse/SOLR-13733
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-13733.patch
>
>
> Most public {{org.apache.solr.metrics}} classes already have class-level
> javadocs and adding javadocs for the few that don't have them yet could be a
> step towards {{ant precommit}} requiring(\*) javadocs (class-level only, for
> public classes, for org.apache.solr.metrics) selectively on an opt-in basis.
> (\*) = e.g. via {{solr/build.xml}} content like this:
> {code}
> dir="${javadoc.dir}/solr-core/org/apache/solr/metrics" level="class"/>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13736) Reduce code duplication in TestPolicy.testNodeLostMultipleReplica
[ https://issues.apache.org/jira/browse/SOLR-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922605#comment-16922605 ] ASF subversion and git services commented on SOLR-13736: Commit ca0f289f375e7480a298b49049af23fcd865474f in lucene-solr's branch refs/heads/branch_8x from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ca0f289 ] SOLR-13736: reduce code duplication in TestPolicy.testNodeLostMultipleReplica > Reduce code duplication in TestPolicy.testNodeLostMultipleReplica > - > > Key: SOLR-13736 > URL: https://issues.apache.org/jira/browse/SOLR-13736 > Project: Solr > Issue Type: Test >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-13736.patch > > > Splitting this refactor out from the SOLR-13240 changes in which it is > currently included. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch, > SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13736) Reduce code duplication in TestPolicy.testNodeLostMultipleReplica
[ https://issues.apache.org/jira/browse/SOLR-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922594#comment-16922594 ] ASF subversion and git services commented on SOLR-13736: Commit 5204d0f963c6b9678c8f58fadc8d1a1ae7b71130 in lucene-solr's branch refs/heads/master from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5204d0f ] SOLR-13736: reduce code duplication in TestPolicy.testNodeLostMultipleReplica > Reduce code duplication in TestPolicy.testNodeLostMultipleReplica > - > > Key: SOLR-13736 > URL: https://issues.apache.org/jira/browse/SOLR-13736 > Project: Solr > Issue Type: Test >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-13736.patch > > > Splitting this refactor out from the SOLR-13240 changes in which it is > currently included. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13733) add more class-level javadocs for public org.apache.solr.metrics classes
[
https://issues.apache.org/jira/browse/SOLR-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922593#comment-16922593
]
ASF subversion and git services commented on SOLR-13733:
Commit 6f12075e9aea61b752bd2fdcbb2a4eb3ade17f31 in lucene-solr's branch
refs/heads/master from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6f12075 ]
SOLR-13733: add class-level javadocs for 4 org.apache.solr.metrics classes
> add more class-level javadocs for public org.apache.solr.metrics classes
>
>
> Key: SOLR-13733
> URL: https://issues.apache.org/jira/browse/SOLR-13733
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-13733.patch
>
>
> Most public {{org.apache.solr.metrics}} classes already have class-level
> javadocs and adding javadocs for the few that don't have them yet could be a
> step towards {{ant precommit}} requiring(\*) javadocs (class-level only, for
> public classes, for org.apache.solr.metrics) selectively on an opt-in basis.
> (\*) = e.g. via {{solr/build.xml}} content like this:
> {code}
> dir="${javadoc.dir}/solr-core/org/apache/solr/metrics" level="class"/>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant commented on issue #828: LUCENE-8753: UniformSplitPostingsFormat
bruno-roustant commented on issue #828: LUCENE-8753: UniformSplitPostingsFormat URL: https://github.com/apache/lucene-solr/pull/828#issuecomment-527941283 I added a new commit to integrate SharedTerms (same as PR#676 now obsolete). It also fixes the lack of BlockEncoder for the FST. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922559#comment-16922559
]
Michael McCandless commented on LUCENE-8962:
Thanks [~dsmiley].
That sounds like a nice improvement to TMP, but I would want to match the more
aggressive merging of those tiny segments w/ a refresh or commit, not run them
in general since I think for pure indexing that'd hurt indexing throughput. I
think the tricky part of this change is fixing {{IndexWriter}} refresh or
commit to let the merge policy know it should now aggressively merge small
segments, within a time or total size budget or something, while the
refresh/commit operation waits, so that the returned segments have been merged,
even while (concurrently) new segments are flushed.
Synchronous merges in merge scheduler sound interesting for this use case –
maybe open a separate issue for that?
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13677) All Metrics Gauges should be unregistered by the objects that registered them
[ https://issues.apache.org/jira/browse/SOLR-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki updated SOLR-13677: - Fix Version/s: 8.3 > All Metrics Gauges should be unregistered by the objects that registered them > - > > Key: SOLR-13677 > URL: https://issues.apache.org/jira/browse/SOLR-13677 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: metrics >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Blocker > Fix For: 8.3 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > The life cycle of Metrics producers are managed by the core (mostly). So, if > the lifecycle of the object is different from that of the core itself, these > objects will never be unregistered from the metrics registry. This will lead > to memory leaks -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13677) All Metrics Gauges should be unregistered by the objects that registered them
[ https://issues.apache.org/jira/browse/SOLR-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922453#comment-16922453 ] Andrzej Bialecki commented on SOLR-13677: -- I marked this as blocker for 8.3 to make sure the changes already committed are fixed before the release. > All Metrics Gauges should be unregistered by the objects that registered them > - > > Key: SOLR-13677 > URL: https://issues.apache.org/jira/browse/SOLR-13677 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: metrics >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Blocker > Fix For: 8.3 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > The life cycle of Metrics producers are managed by the core (mostly). So, if > the lifecycle of the object is different from that of the core itself, these > objects will never be unregistered from the metrics registry. This will lead > to memory leaks -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8241) Evaluate W-TinyLfu cache
[ https://issues.apache.org/jira/browse/SOLR-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki updated SOLR-8241: Fix Version/s: master (9.0) > Evaluate W-TinyLfu cache > > > Key: SOLR-8241 > URL: https://issues.apache.org/jira/browse/SOLR-8241 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Ben Manes >Assignee: Andrzej Bialecki >Priority: Major > Fix For: master (9.0) > > Attachments: SOLR-8241.patch, SOLR-8241.patch, SOLR-8241.patch, > proposal.patch > > > SOLR-2906 introduced an LFU cache and in-progress SOLR-3393 makes it O(1). > The discussions seem to indicate that the higher hit rate (vs LRU) is offset > by the slower performance of the implementation. An original goal appeared to > be to introduce ARC, a patented algorithm that uses ghost entries to retain > history information. > My analysis of Window TinyLfu indicates that it may be a better option. It > uses a frequency sketch to compactly estimate an entry's popularity. It uses > LRU to capture recency and operate in O(1) time. When using available > academic traces the policy provides a near optimal hit rate regardless of the > workload. > I'm getting ready to release the policy in Caffeine, which Solr already has a > dependency on. But, the code is fairly straightforward and a port into Solr's > caches instead is a pragmatic alternative. More interesting is what the > impact would be in Solr's workloads and feedback on the policy's design. > https://github.com/ben-manes/caffeine/wiki/Efficiency -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13677) All Metrics Gauges should be unregistered by the objects that registered them
[ https://issues.apache.org/jira/browse/SOLR-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki updated SOLR-13677: - Priority: Blocker (was: Major) > All Metrics Gauges should be unregistered by the objects that registered them > - > > Key: SOLR-13677 > URL: https://issues.apache.org/jira/browse/SOLR-13677 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: metrics >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Blocker > Time Spent: 3h 20m > Remaining Estimate: 0h > > The life cycle of Metrics producers are managed by the core (mostly). So, if > the lifecycle of the object is different from that of the core itself, these > objects will never be unregistered from the metrics registry. This will lead > to memory leaks -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-8241) Evaluate W-TinyLfu cache
[ https://issues.apache.org/jira/browse/SOLR-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki reassigned SOLR-8241: --- Assignee: Andrzej Bialecki > Evaluate W-TinyLfu cache > > > Key: SOLR-8241 > URL: https://issues.apache.org/jira/browse/SOLR-8241 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Ben Manes >Assignee: Andrzej Bialecki >Priority: Major > Attachments: SOLR-8241.patch, SOLR-8241.patch, SOLR-8241.patch, > proposal.patch > > > SOLR-2906 introduced an LFU cache and in-progress SOLR-3393 makes it O(1). > The discussions seem to indicate that the higher hit rate (vs LRU) is offset > by the slower performance of the implementation. An original goal appeared to > be to introduce ARC, a patented algorithm that uses ghost entries to retain > history information. > My analysis of Window TinyLfu indicates that it may be a better option. It > uses a frequency sketch to compactly estimate an entry's popularity. It uses > LRU to capture recency and operate in O(1) time. When using available > academic traces the policy provides a near optimal hit rate regardless of the > workload. > I'm getting ready to release the policy in Caffeine, which Solr already has a > dependency on. But, the code is fairly straightforward and a port into Solr's > caches instead is a pragmatic alternative. More interesting is what the > impact would be in Solr's workloads and feedback on the policy's design. > https://github.com/ben-manes/caffeine/wiki/Efficiency -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on issue #838: SOLR-13705 Double-checked locking bug is fixed.
dsmiley commented on issue #838: SOLR-13705 Double-checked locking bug is fixed. URL: https://github.com/apache/lucene-solr/pull/838#issuecomment-527874693 +1 to the code you have here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9658) Caches should have an optional way to clean if idle for 'x' mins
[
https://issues.apache.org/jira/browse/SOLR-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922432#comment-16922432
]
David Smiley commented on SOLR-9658:
Sure; I don't mean to stand in your way.
> Caches should have an optional way to clean if idle for 'x' mins
>
>
> Key: SOLR-9658
> URL: https://issues.apache.org/jira/browse/SOLR-9658
> Project: Solr
> Issue Type: New Feature
>Reporter: Noble Paul
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch,
> SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch
>
>
> If a cache is idle for long, it consumes precious memory. It should be
> configurable to clear the cache if it was not accessed for 'x' secs. The
> cache configuration can have an extra config {{maxIdleTime}} . if we wish it
> to the cleaned after 10 mins of inactivity set it to {{maxIdleTime=600}}.
> [~dragonsinth] would it be a solution for the memory leak you mentioned?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant commented on a change in pull request #797: LUCENE-8921: IndexSearcher.termStatistics requires docFreq totalTermFreq instead of TermStates
bruno-roustant commented on a change in pull request #797: LUCENE-8921:
IndexSearcher.termStatistics requires docFreq totalTermFreq instead of
TermStates
URL: https://github.com/apache/lucene-solr/pull/797#discussion_r320722038
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -861,17 +861,17 @@ public String toString() {
/**
* Returns {@link TermStatistics} for a term, or {@code null} if
- * the term does not exist.
+ * the term does not exist (if docFreq == 0).
*
* This can be overridden for example, to return a term's statistics
* across a distributed collection.
* @lucene.experimental
*/
- public TermStatistics termStatistics(Term term, TermStates context) throws
IOException {
-if (context.docFreq() == 0) {
+ public TermStatistics termStatistics(Term term, int docFreq, long
totalTermFreq) throws IOException {
+if (docFreq == 0) {
Review comment:
+1 also
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #797: LUCENE-8921: IndexSearcher.termStatistics requires docFreq totalTermFreq instead of TermStates
jpountz commented on a change in pull request #797: LUCENE-8921:
IndexSearcher.termStatistics requires docFreq totalTermFreq instead of
TermStates
URL: https://github.com/apache/lucene-solr/pull/797#discussion_r320717177
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -861,17 +861,17 @@ public String toString() {
/**
* Returns {@link TermStatistics} for a term, or {@code null} if
- * the term does not exist.
+ * the term does not exist (if docFreq == 0).
*
* This can be overridden for example, to return a term's statistics
* across a distributed collection.
* @lucene.experimental
*/
- public TermStatistics termStatistics(Term term, TermStates context) throws
IOException {
-if (context.docFreq() == 0) {
+ public TermStatistics termStatistics(Term term, int docFreq, long
totalTermFreq) throws IOException {
+if (docFreq == 0) {
Review comment:
+1 to your idea of deprecating + marking final and introducing the new API
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922407#comment-16922407
]
ASF subversion and git services commented on SOLR-13105:
Commit cf7f77b77a1f7294dfa7913179070d2ad8446f39 in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=cf7f77b ]
SOLR-13105: Revamp simulations docs 16
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #797: LUCENE-8921: IndexSearcher.termStatistics requires docFreq totalTermFreq instead of TermStates
dsmiley commented on a change in pull request #797: LUCENE-8921:
IndexSearcher.termStatistics requires docFreq totalTermFreq instead of
TermStates
URL: https://github.com/apache/lucene-solr/pull/797#discussion_r320713838
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -861,17 +861,17 @@ public String toString() {
/**
* Returns {@link TermStatistics} for a term, or {@code null} if
- * the term does not exist.
+ * the term does not exist (if docFreq == 0).
*
* This can be overridden for example, to return a term's statistics
* across a distributed collection.
* @lucene.experimental
*/
- public TermStatistics termStatistics(Term term, TermStates context) throws
IOException {
-if (context.docFreq() == 0) {
+ public TermStatistics termStatistics(Term term, int docFreq, long
totalTermFreq) throws IOException {
+if (docFreq == 0) {
Review comment:
There is still the issue of subclassing but the current one could be marked
final. Again, this is lucene.experimental code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #797: LUCENE-8921: IndexSearcher.termStatistics requires docFreq totalTermFreq instead of TermStates
dsmiley commented on a change in pull request #797: LUCENE-8921:
IndexSearcher.termStatistics requires docFreq totalTermFreq instead of
TermStates
URL: https://github.com/apache/lucene-solr/pull/797#discussion_r320713047
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -861,17 +861,17 @@ public String toString() {
/**
* Returns {@link TermStatistics} for a term, or {@code null} if
- * the term does not exist.
+ * the term does not exist (if docFreq == 0).
*
* This can be overridden for example, to return a term's statistics
* across a distributed collection.
* @lucene.experimental
*/
- public TermStatistics termStatistics(Term term, TermStates context) throws
IOException {
-if (context.docFreq() == 0) {
+ public TermStatistics termStatistics(Term term, int docFreq, long
totalTermFreq) throws IOException {
+if (docFreq == 0) {
Review comment:
I think in 8x we could keep the old method signature, mark it deprecated,
and have it call this new one. It's marked lucene.experimental, after all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-repro-Java11 - Build # 418 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro-Java11/418/ [...truncated 31 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-Tests-master/3676/consoleText [repro] Revision: d1a4d1352538a0d967a12686ca903453d10c48c9 [repro] Repro line: ant test -Dtestcase=TestCloudJSONFacetSKG -Dtests.method=testRandom -Dtests.seed=5D445E53BDFA995 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=twq -Dtests.timezone=SystemV/AST4ADT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 26804a069b929ac53e3e9b4c0431bddc72a6807b [repro] git fetch [repro] git checkout d1a4d1352538a0d967a12686ca903453d10c48c9 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestCloudJSONFacetSKG [repro] ant compile-test [...truncated 3315 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestCloudJSONFacetSKG" -Dtests.showOutput=onerror -Dtests.seed=5D445E53BDFA995 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=twq -Dtests.timezone=SystemV/AST4ADT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 62663 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 4/5 failed: org.apache.solr.search.facet.TestCloudJSONFacetSKG [repro] git checkout 26804a069b929ac53e3e9b4c0431bddc72a6807b [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 6 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Policies Around Posting On Wiki
No policies as far as I know, just throw up a page and ask for review I guess. When we get around to making the new lucene developer guide somewhere we could define such policy. I think it is a good idea to find one way to do more thorough design review and also design documentation. Design docs could go in the new Asciidoc developer guide I guess, but design draft for review could start its life in Confluence... -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 4. sep. 2019 kl. 09:34 skrev Atri Sharma : > > Hi Folks, > > Do we have a policy around sharing design details of a feature on the > Wiki? I am working something which I would like design review on, as > well document it for future reference. > > ./Atri > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org >
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922370#comment-16922370
]
ASF subversion and git services commented on SOLR-13105:
Commit d77cee780e7f4c9f822b7108697ef6b4875a5a8a in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d77cee7 ]
SOLR-13105: Revamp simulations docs 15
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922369#comment-16922369
]
ASF subversion and git services commented on SOLR-13105:
Commit 46429a2e2c020c653867d6c52aab8a0b0dd337dc in lucene-solr's branch
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=46429a2 ]
SOLR-13105: Revamp simulations docs 14
> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
> Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png,
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and
> Math Expressions. This ticket will create a visual guide to Solr Math
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin*
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize*
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable
> functions. These functions provide a system for searching, transforming,
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>
> A few sample visualizations are attached to the ticket.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] thomaswoeckinger opened a new pull request #855: SOLR-13739: Improve performance on huge schema updates
thomaswoeckinger opened a new pull request #855: SOLR-13739: Improve performance on huge schema updates URL: https://github.com/apache/lucene-solr/pull/855 Affects schemas containing ResourceLoaderAware components, increases creation of a new collection with a huge schema by a factor 60 to 100. # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute. - [x] I have developed this patch against the `master` branch. - [x] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on issue #854: Shared PQ Based Early Termination for Concurrent Search
atris commented on issue #854: Shared PQ Based Early Termination for Concurrent Search URL: https://github.com/apache/lucene-solr/pull/854#issuecomment-527823824 cc @jpountz. This PR builds on top of #831 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] atris opened a new pull request #854: Shared PQ Based Early Termination for Concurrent Search
atris opened a new pull request #854: Shared PQ Based Early Termination for Concurrent Search URL: https://github.com/apache/lucene-solr/pull/854 NOCOMMIT. This is a WIP PR which implements a shared PQ based early termination. Each collector collects independently into a thread local PQ and updates a global count of hits (which will be refactored post merging of LUCENE-8939). Once enough hits are accumulated, a global PQ is populated by all collectors and then the global PQ serves as the benchmark to filter further hits. Several optimizations have been performed, such as non blocking building of global PQ, no reduce operation performed during CollectorManager.reduce but rather, returning results directly from the global PQ. I need some eyes on the overall logic, and especially at two points: 1) Across segment values comparison and 2) Local to global DocID mapping during interactions of thread local PQs and global PQ. I am pretty sure there is a bug in either of the two, so would really appreciate a deep look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8963) Allow Collectors To "Publish" If They Can Be Used In Concurrent Search
[ https://issues.apache.org/jira/browse/LUCENE-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922322#comment-16922322 ] Atri Sharma commented on LUCENE-8963: - Yeah, I agree. My only gripe is that in case a collector is not really reducible or has some semantic constraints against concurrency, we do not provide any defense against getting into an unknown state. Maybe it is not an engine problem but more of a user issue – but I wanted to raise this point and see if we have any thoughts about this. > Allow Collectors To "Publish" If They Can Be Used In Concurrent Search > -- > > Key: LUCENE-8963 > URL: https://issues.apache.org/jira/browse/LUCENE-8963 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Atri Sharma >Priority: Major > > There is an implied assumption today that all we need to run a query > concurrently is a CollectorManager implementation. While that is true, there > might be some corner cases where a Collector's semantics do not allow it to > be concurrently executed (think of ES's aggregates). If a user manages to > write a CollectorManager with a Collector that is not really concurrent > friendly, we could end up in an undefined state. > > This Jira is more of a rhetorical discussion, and to explore if we should > allow Collectors to implement an API which simply returns a boolean > signifying if a Collector is parallel ready or not. The default would be > true, until a Collector explicitly overrides it? -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8963) Allow Collectors To "Publish" If They Can Be Used In Concurrent Search
[ https://issues.apache.org/jira/browse/LUCENE-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922304#comment-16922304 ] Adrien Grand commented on LUCENE-8963: -- I don't think this would solve any problem? Collectors can only run from a single thread anyway, and all collectors could have a CollectorManager provided that there is a way that the results that they produce can be merged? > Allow Collectors To "Publish" If They Can Be Used In Concurrent Search > -- > > Key: LUCENE-8963 > URL: https://issues.apache.org/jira/browse/LUCENE-8963 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Atri Sharma >Priority: Major > > There is an implied assumption today that all we need to run a query > concurrently is a CollectorManager implementation. While that is true, there > might be some corner cases where a Collector's semantics do not allow it to > be concurrently executed (think of ES's aggregates). If a user manages to > write a CollectorManager with a Collector that is not really concurrent > friendly, we could end up in an undefined state. > > This Jira is more of a rhetorical discussion, and to explore if we should > allow Collectors to implement an API which simply returns a boolean > signifying if a Collector is parallel ready or not. The default would be > true, until a Collector explicitly overrides it? -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13739) Managed resource observers have to be added only once
[ https://issues.apache.org/jira/browse/SOLR-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922296#comment-16922296 ] Thomas Wöckinger commented on SOLR-13739: - The improvement on our side results in a creation time of ~30 seconds instead of 30 minutes > Managed resource observers have to be added only once > - > > Key: SOLR-13739 > URL: https://issues.apache.org/jira/browse/SOLR-13739 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Server >Affects Versions: 8.0, 8.1, 8.2, 8.1.1 >Reporter: Thomas Wöckinger >Priority: Major > Labels: easyfix, performance > > On huge schema modifications, mostly happen due to creation of a new > collection, the same observer instance of an ResourceLoaderAware component is > added again and again. > This leads to a runtime behaviour of n²/2 where n is is the number of schema > operation multiplied by ResourceLoaderAware components instead of the number > of containing ResourceLoaderAware components. > E.g. If you have 1000 schema operations and 2 ResourceLoaderAware components > this leads to 50 operations instead of 2000. > Even worse the corresponding resource is registered again and again, which > can take some time e.g. ManagedSynonymGraphFilterFactory needs about 5s on > each call (depending on the size of synonyms). -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on issue #648: LUCENE-8766: Add Luwak as a lucene module
jpountz commented on issue #648: LUCENE-8766: Add Luwak as a lucene module URL: https://github.com/apache/lucene-solr/pull/648#issuecomment-527806855 This has been merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13739) Managed resource observers have to be added only once
Thomas Wöckinger created SOLR-13739: --- Summary: Managed resource observers have to be added only once Key: SOLR-13739 URL: https://issues.apache.org/jira/browse/SOLR-13739 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Server Affects Versions: 8.1.1, 8.2, 8.1, 8.0 Reporter: Thomas Wöckinger On huge schema modifications, mostly happen due to creation of a new collection, the same observer instance of an ResourceLoaderAware component is added again and again. This leads to a runtime behaviour of n²/2 where n is is the number of schema operation multiplied by ResourceLoaderAware components instead of the number of containing ResourceLoaderAware components. E.g. If you have 1000 schema operations and 2 ResourceLoaderAware components this leads to 50 operations instead of 2000. Even worse the corresponding resource is registered again and again, which can take some time e.g. ManagedSynonymGraphFilterFactory needs about 5s on each call (depending on the size of synonyms). -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz closed pull request #648: LUCENE-8766: Add Luwak as a lucene module
jpountz closed pull request #648: LUCENE-8766: Add Luwak as a lucene module URL: https://github.com/apache/lucene-solr/pull/648 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13738) RequestHandlerBase ... ClassCastException: class ... .lucene.search.IndexSearcher cannot be cast to class ... .solr.search.SolrIndexSearcher ...
[
https://issues.apache.org/jira/browse/SOLR-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jochen Barth updated SOLR-13738:
Description:
Mikhail Khludnev said, this is a bug;
Here the complete error message for the Query below:
Just tested wirth 8.1.1: works.
{quote}
2019-08-30 12:40:40.476 ERROR (qtp2116511124-65) [ x:Suchindex]
o.a.s.h.RequestHandlerBase java.lang.ClassCastException: class
org.apache.lucene.search.IndexSearcher cannot be cast to class
org.apache.solr.search.SolrIndexSearcher (or
g.apache.lucene.search.IndexSearcher and
org.apache.solr.search.SolrIndexSearcher are in unnamed module of loader
org.eclipse.jetty.webapp.WebAppClassLoader @5ed190be)
at
org.apache.solr.search.join.GraphQuery.createWeight(GraphQuery.java:115)
at
org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumsWeightMatcher(FieldOffsetStrategy.java:137)
at
org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumFromReader(FieldOffsetStrategy.java:74)
at
org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.getOffsetsEnum(MemoryIndexOffsetStrategy.java:110)
at
org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:76)
at
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:641)
at
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:510)
at
org.apache.solr.highlight.UnifiedSolrHighlighter.doHighlighting(UnifiedSolrHighlighter.java:149)
at
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:171)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at
[jira] [Created] (SOLR-13738) RequestHandlerBase ... ClassCastException: class ... .lucene.search.IndexSearcher cannot be cast to class ... .solr.search.SolrIndexSearcher ...
Jochen Barth created SOLR-13738:
---
Summary: RequestHandlerBase ... ClassCastException: class ...
.lucene.search.IndexSearcher cannot be cast to class ...
.solr.search.SolrIndexSearcher ...
Key: SOLR-13738
URL: https://issues.apache.org/jira/browse/SOLR-13738
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 8.2
Reporter: Jochen Barth
Mikhail Khludnev said, this is a bug;
Here the complete error message for the Query below:
Just tested wirth 8.1.1: works.
{quote}
2019-08-30 12:40:40.476 ERROR (qtp2116511124-65) [ x:Suchindex]
o.a.s.h.RequestHandlerBase java.lang.ClassCastException: class
org.apache.lucene.search.IndexSearcher cannot be cast to class
org.apache.solr.search.SolrIndexSearcher (or
g.apache.lucene.search.IndexSearcher and
org.apache.solr.search.SolrIndexSearcher are in unnamed module of loader
org.eclipse.jetty.webapp.WebAppClassLoader @5ed190be)
at
org.apache.solr.search.join.GraphQuery.createWeight(GraphQuery.java:115)
at
org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumsWeightMatcher(FieldOffsetStrategy.java:137)
at
org.apache.lucene.search.uhighlight.FieldOffsetStrategy.createOffsetsEnumFromReader(FieldOffsetStrategy.java:74)
at
org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.getOffsetsEnum(MemoryIndexOffsetStrategy.java:110)
at
org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:76)
at
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:641)
at
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:510)
at
org.apache.solr.highlight.UnifiedSolrHighlighter.doHighlighting(UnifiedSolrHighlighter.java:149)
at
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:171)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
[jira] [Created] (LUCENE-8963) Allow Collectors To "Publish" If They Can Be Used In Concurrent Search
Atri Sharma created LUCENE-8963: --- Summary: Allow Collectors To "Publish" If They Can Be Used In Concurrent Search Key: LUCENE-8963 URL: https://issues.apache.org/jira/browse/LUCENE-8963 Project: Lucene - Core Issue Type: Improvement Reporter: Atri Sharma There is an implied assumption today that all we need to run a query concurrently is a CollectorManager implementation. While that is true, there might be some corner cases where a Collector's semantics do not allow it to be concurrently executed (think of ES's aggregates). If a user manages to write a CollectorManager with a Collector that is not really concurrent friendly, we could end up in an undefined state. This Jira is more of a rhetorical discussion, and to explore if we should allow Collectors to implement an API which simply returns a boolean signifying if a Collector is parallel ready or not. The default would be true, until a Collector explicitly overrides it? -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9658) Caches should have an optional way to clean if idle for 'x' mins
[
https://issues.apache.org/jira/browse/SOLR-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922269#comment-16922269
]
Andrzej Bialecki commented on SOLR-9658:
-
[~dsmiley] thanks, I wasn't aware of that Jira. This makes sense - I'll look
into that issue.
Still, even if we eventually replace our cache implementations with Caffeine
we're probably going to do this in 9.0 and not in 8.x, so the improvement
proposed in this issue is still applicable.
> Caches should have an optional way to clean if idle for 'x' mins
>
>
> Key: SOLR-9658
> URL: https://issues.apache.org/jira/browse/SOLR-9658
> Project: Solr
> Issue Type: New Feature
>Reporter: Noble Paul
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch,
> SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch
>
>
> If a cache is idle for long, it consumes precious memory. It should be
> configurable to clear the cache if it was not accessed for 'x' secs. The
> cache configuration can have an extra config {{maxIdleTime}} . if we wish it
> to the cleaned after 10 mins of inactivity set it to {{maxIdleTime=600}}.
> [~dragonsinth] would it be a solution for the memory leak you mentioned?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8150) Remove references to segments.gen.
[
https://issues.apache.org/jira/browse/LUCENE-8150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-8150:
-
Fix Version/s: master (9.0)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove references to segments.gen.
> --
>
> Key: LUCENE-8150
> URL: https://issues.apache.org/jira/browse/LUCENE-8150
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Fix For: master (9.0)
>
> Attachments: LUCENE-8150.patch, LUCENE-8150.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This was the way we wrote pending segment files before we switch to
> {{pending_segments_N}} in LUCENE-5925.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8150) Remove references to segments.gen.
[
https://issues.apache.org/jira/browse/LUCENE-8150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922251#comment-16922251
]
ASF subversion and git services commented on LUCENE-8150:
-
Commit 26804a069b929ac53e3e9b4c0431bddc72a6807b in lucene-solr's branch
refs/heads/master from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=26804a0 ]
LUCENE-8150: Remove references to `segments.gen`. (#765)
This file isn't used anymore since 4.0, so I tried to contain references to
`segments.gen` to the minimum that is required to get the right exception when
opening a too old index.
> Remove references to segments.gen.
> --
>
> Key: LUCENE-8150
> URL: https://issues.apache.org/jira/browse/LUCENE-8150
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8150.patch, LUCENE-8150.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This was the way we wrote pending segment files before we switch to
> {{pending_segments_N}} in LUCENE-5925.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz merged pull request #765: LUCENE-8150: Remove references to `segments.gen`.
jpountz merged pull request #765: LUCENE-8150: Remove references to `segments.gen`. URL: https://github.com/apache/lucene-solr/pull/765 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on issue #769: LUCENE-8905: Better Error Handling For Illegal Arguments
jpountz commented on issue #769: LUCENE-8905: Better Error Handling For Illegal Arguments URL: https://github.com/apache/lucene-solr/pull/769#issuecomment-527786074 > what is the expected behaviour when start > size? Should we error out, or return a null topdocs (as we do today), or normalize the start value to be 0? I don't think we should error, because users have no way to know up-front that the query they ran has less than `start` hits. So let's keep returning empty hits? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on issue #831: LUCENE-8949: Allow LeafFieldComparators to publish Feature Values
jpountz commented on issue #831: LUCENE-8949: Allow LeafFieldComparators to publish Feature Values URL: https://github.com/apache/lucene-solr/pull/831#issuecomment-527784806 I can imagine it be useful, but on the other hand this increases the API surface area of comparators while we have been talking about simplifying them, and like Mike pointed out this is a bit trappy as this sounds like a harmless method call, but this is actually costly for string or geo-distance comparators. So I'd like to better understand what benefits it might bring before making the call of adding this new API. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on issue #775: LUCENE-8910: upgrade to icu 62.1 must be completed
jpountz commented on issue #775: LUCENE-8910: upgrade to icu 62.1 must be completed URL: https://github.com/apache/lucene-solr/pull/775#issuecomment-527782071 @rmuir This looks good to me but would you mind having a second look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #797: LUCENE-8921: IndexSearcher.termStatistics requires docFreq totalTermFreq instead of TermStates
jpountz commented on a change in pull request #797: LUCENE-8921:
IndexSearcher.termStatistics requires docFreq totalTermFreq instead of
TermStates
URL: https://github.com/apache/lucene-solr/pull/797#discussion_r320612366
##
File path: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
##
@@ -861,17 +861,17 @@ public String toString() {
/**
* Returns {@link TermStatistics} for a term, or {@code null} if
- * the term does not exist.
+ * the term does not exist (if docFreq == 0).
*
* This can be overridden for example, to return a term's statistics
* across a distributed collection.
* @lucene.experimental
*/
- public TermStatistics termStatistics(Term term, TermStates context) throws
IOException {
-if (context.docFreq() == 0) {
+ public TermStatistics termStatistics(Term term, int docFreq, long
totalTermFreq) throws IOException {
+if (docFreq == 0) {
Review comment:
That works for me, but then let's only merge this to master? My concern is
that this IndexSearcher method might not only be used internally but also by
Lucene users.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Policies Around Posting On Wiki
Hi Folks, Do we have a policy around sharing design details of a feature on the Wiki? I am working something which I would like design review on, as well document it for future reference. ./Atri - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] atris commented on issue #831: LUCENE-8949: Allow LeafFieldComparators to publish Feature Values
atris commented on issue #831: LUCENE-8949: Allow LeafFieldComparators to publish Feature Values URL: https://github.com/apache/lucene-solr/pull/831#issuecomment-527779942 > I appreciate how you are trying to break a large change into smaller changes, but for this one I'd like to see how you would leverage this new API to implement the optimizations that you mentioned? Thanks for your comments, @jpountz I abstracted this specific change out because of the reason you mentioned, and because I believe that this change independently allows us some new functionality, such as comparison by values across multiple segments, something that we do not allow today. Does the second functionality seem a useful enough thing to support in itself? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
