[jira] [Updated] (SOLR-13806) SolrJ QueryResponse._explainMap is incorrectly typed
[
https://issues.apache.org/jira/browse/SOLR-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-13806:
Labels: newdev (was: )
> SolrJ QueryResponse._explainMap is incorrectly typed
>
>
> Key: SOLR-13806
> URL: https://issues.apache.org/jira/browse/SOLR-13806
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.3
>Reporter: Andrzej Bialecki
>Priority: Major
> Labels: newdev
>
> This field, and the corresponding getter, and the code that extracts debug
> information from the NamedList response in {{extractDebugInfo}}, all use a
> {{Map}} type.
> However, when {{debug.explain.structured=true}} is used the values returned
> in response are not String-s, instead they are {{SimpleOrderedMap}}-s. This
> causes the following exception to be thrown:
> {code}
> java.lang.ClassCastException: class
> org.apache.solr.common.util.SimpleOrderedMap cannot be cast to class
> java.lang.String (org.apache.solr.common.util.SimpleOrderedMap is in unnamed
> module of loader 'app'; java.lang.String is in module java.base of loader
> 'bootstrap')
> at
> __randomizedtesting.SeedInfo.seed([1D6FB4036A639051:173F5FF19E860E6F]:0)
> at
> org.apache.solr.client.solrj.response.QueryResponse.extractDebugInfo(QueryResponse.java:246)
> at
> org.apache.solr.client.solrj.response.QueryResponse.setResponse(QueryResponse.java:143)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
> at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:987)
> ...
> {code}
> The simple fix is to change the type of this field to {{Map}}
> but this would change the public API of {{QueryResponse.getExplainMap()}} in
> incompatible way. Still, I would argue it's worth to change it - AFAIK this
> getter is not used anywhere in the Solr's codebase, and the change makes it
> more consistent with other getters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13842) Remove wt=json from Implicit API definition's defaults
[
https://issues.apache.org/jira/browse/SOLR-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-13842:
Labels: newdev (was: )
> Remove wt=json from Implicit API definition's defaults
> --
>
> Key: SOLR-13842
> URL: https://issues.apache.org/jira/browse/SOLR-13842
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Priority: Minor
> Labels: newdev
>
> From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be
> removed from implicit API definitions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960506#comment-16960506
]
Gus Heck commented on SOLR-13749:
-
After some fooling around with this I found that arbitrary
{color:#de350b}POST{color} requests can be sent via the url parameter like this
{code:none}
{!xcjf solrUrl="http://localhost:8080/login#; collection="results"
from="driverId_s" to="id" v="laps_i:[100 TO *]"}
{code}
or to pester someone's mail server
{code:none}
{!xcjf solrUrl=\"http://localhost:25/login#\; collection=\"results\"
from=\"driverId_s\" to=\"id\" v=\"laps_i:[100 TO *]\"}
{code}
creating log entries like this in exim4:
{code:java}
2019-10-27 01:17:17 SMTP protocol synchronization error (input sent without
waiting for greeting): rejected connection from H=ns-l1 [127.0.0.1] input="POST
/login HTTP/1.1\r\nContent-Ty"
2019-10-27 01:17:17 SMTP protocol synchronization error (input sent without
waiting for greeting): rejected connection from H=ns-l1 [127.0.0.1] input="POST
/login HTTP/1.1\r\nContent-Ty"
2019-10-27 01:17:17 SMTP protocol synchronization error (input sent without
waiting for greeting): rejected connection from H=ns-l1 [127.0.0.1] input="POST
/login HTTP/1.1\r\nContent-Ty"
2019-10-27 01:17:17 SMTP protocol synchronization error (input sent without
waiting for greeting): rejected connection from H=ns-l1 [127.0.0.1] input="POST
/login HTTP/1.1\r\nContent-Ty"
(note the amplification by the number of nodes in the solr cluster...){code}
However, I haven't yet found a way to trick the server into accepting query
parameters (since the ? and & chars break it... But that only fails because of
this dodgy looking exception:
{code:java}
o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
at
org.apache.solr.client.solrj.io.stream.SolrStream.close(SolrStream.java:191)
at
org.apache.solr.search.join.XCJFQuery$XCJFQueryWeight.getDocSet(XCJFQuery.java:301)
at
org.apache.solr.search.join.XCJFQuery$XCJFQueryWeight.scorer(XCJFQuery.java:311)
at org.apache.lucene.search.Weight.bulkScorer(Weight.java:168)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:741)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:516)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:203)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1621)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1437)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
at
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1485)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)
...etc
{code}
Not entirely sure that it isn't a bug that a SolrStream got created without a
client (which is what's null)... and I'm not sure if something getting fixed
later might enable parameters to go through. Also, one could use this to map
visible networks via Solr since responding ports come back with errors quickly
and requests to machines that don't exist spin for a long time
It seems like perhaps there should be a white list of servers this query is
allowed to reach out to so that attackers can't use it to poke and probe
whatever network or apps solr can see. This could possibly be a collection
property, though if we do that we need to be carefull that the value is cached
and watched somewhere, not re-fetched from zk every request. There may be other
good options... Open to ideas of course.
I've not yet fooled the expressions you build with injection attacks but still
working on that ;). You might be safe because there's no let() or parallel() in
play there.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #958: LUCENE-8986: Add asf.yaml to our git repo
dsmiley commented on a change in pull request #958: LUCENE-8986: Add asf.yaml to our git repo URL: https://github.com/apache/lucene-solr/pull/958#discussion_r339324263 ## File path: .asf.yaml ## @@ -0,0 +1,19 @@ +# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories + +github: + description: "Apache Lucene and Solr open-source search software" + homepage: https://lucene.apache.org/ + labels: +- lucene +- solr +- search +- sql +- nosql +- java +- backend +- search-engine Review comment: lets add "information-retrieval" like my proposal on the mailing list had? Do you really think "sql" is a suitable entry in this list? This is one small feature of Solr and it's _very_ limited to the full syntax people expect from SQL. So, no, I think we shouldn't put it in this list. I think "java" is redundant with GitHub's language analysis. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] Pr0methean opened a new pull request #979: SOLR-13783: Add space after comma in NamedList.toString()
Pr0methean opened a new pull request #979: SOLR-13783: Add space after comma in NamedList.toString() URL: https://github.com/apache/lucene-solr/pull/979 # Description NamedList.toString() differs in format from AbstractMap.toString() when the NamedList has 2 or more entries, in that there's no space after each comma between entries. # Solution Added a space after the comma. # Tests Added NamedListTest.testToString(), which covers toString() for the cases with 0, 1 and 2 entries. (The method currently seems to have no test coverage.) # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. **(Existing issue)** - [X] I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [X] I have developed this patch against the `master` branch. - [X] I have run `ant precommit` and the appropriate test suite. - [X] I have added tests for my changes. - [X] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). **(No changes)** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] Pr0methean opened a new pull request #978: SOLR-13207: Handle misuse of < operator
Pr0methean opened a new pull request #978: SOLR-13207: Handle misuse of < operator URL: https://github.com/apache/lucene-solr/pull/978 Traps error that arises when the < operator is used at the end of a query field. Also handles NumberFormatException when the operand isn't a number. # Description When < operator is used with no operand after it (i.e. at the end of the query field) or with a non-numeric operand, an HTTP 500 error arises caused by an ArrayIndexOutOfBoundsException or NumberFormatException respectively. # Solution Throws a SolrException with BAD_REQUEST as the error code when either situation arises, since the query is malformed. # Tests Added 2 cases in SolrPluginUtilsTest.testMinShouldMatchCalculator, for queries "1<" and "1https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. **Using an existing Jira issue instead.** - [X] I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute. **This PR consists entirely of my own code.** - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [X] I have developed this patch against the `master` branch. - [X] I have run `ant precommit` and the appropriate test suite. - [X] I have added tests for my changes. - [X] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). **No changes apply.** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8986) Add asf.yaml to our git repo
[
https://issues.apache.org/jira/browse/LUCENE-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960427#comment-16960427
]
Jan Høydahl commented on LUCENE-8986:
-
!lucene-github-changed.png|width=600!
!disabled.png|width=331!
> Add asf.yaml to our git repo
>
>
> Key: LUCENE-8986
> URL: https://issues.apache.org/jira/browse/LUCENE-8986
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: disabled.png, lucene-github-changed.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Adding a {{asf.yaml}} file to our git repo allows us to control the
> description, link and labels on Lucene-Solr project git page. See
> https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E
> for more.
> I'll post a PR with the suggested change
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8986) Add asf.yaml to our git repo
[
https://issues.apache.org/jira/browse/LUCENE-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated LUCENE-8986:
Attachment: disabled.png
> Add asf.yaml to our git repo
>
>
> Key: LUCENE-8986
> URL: https://issues.apache.org/jira/browse/LUCENE-8986
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: disabled.png, lucene-github-changed.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Adding a {{asf.yaml}} file to our git repo allows us to control the
> description, link and labels on Lucene-Solr project git page. See
> https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E
> for more.
> I'll post a PR with the suggested change
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8986) Add asf.yaml to our git repo
[
https://issues.apache.org/jira/browse/LUCENE-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated LUCENE-8986:
Attachment: lucene-github-changed.png
> Add asf.yaml to our git repo
>
>
> Key: LUCENE-8986
> URL: https://issues.apache.org/jira/browse/LUCENE-8986
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-github-changed.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Adding a {{asf.yaml}} file to our git repo allows us to control the
> description, link and labels on Lucene-Solr project git page. See
> https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E
> for more.
> I'll post a PR with the suggested change
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8986) Add asf.yaml to our git repo
[
https://issues.apache.org/jira/browse/LUCENE-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl resolved LUCENE-8986.
-
Resolution: Fixed
Merged
> Add asf.yaml to our git repo
>
>
> Key: LUCENE-8986
> URL: https://issues.apache.org/jira/browse/LUCENE-8986
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-github-changed.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Adding a {{asf.yaml}} file to our git repo allows us to control the
> description, link and labels on Lucene-Solr project git page. See
> https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E
> for more.
> I'll post a PR with the suggested change
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy merged pull request #958: LUCENE-8986: Add asf.yaml to our git repo
janhoy merged pull request #958: LUCENE-8986: Add asf.yaml to our git repo URL: https://github.com/apache/lucene-solr/pull/958 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8986) Add asf.yaml to our git repo
[
https://issues.apache.org/jira/browse/LUCENE-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960422#comment-16960422
]
Jan Høydahl commented on LUCENE-8986:
-
Latest commit also restricts merge button to squash merge.
> Add asf.yaml to our git repo
>
>
> Key: LUCENE-8986
> URL: https://issues.apache.org/jira/browse/LUCENE-8986
> Project: Lucene - Core
> Issue Type: Task
> Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Adding a {{asf.yaml}} file to our git repo allows us to control the
> description, link and labels on Lucene-Solr project git page. See
> https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E
> for more.
> I'll post a PR with the suggested change
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #966: LUCENE-9024 Optimize IntroSelector to use median of medians
jpountz commented on a change in pull request #966: LUCENE-9024 Optimize
IntroSelector to use median of medians
URL: https://github.com/apache/lucene-solr/pull/966#discussion_r339307490
##
File path: lucene/core/src/java/org/apache/lucene/util/IntroSelector.java
##
@@ -36,29 +36,30 @@ int slowSelect(int from, int to, int k) {
return medianOfMediansSelect(from, to-1, k);
}
- int medianOfMediansSelect(int from, int to, int k) {
+ int medianOfMediansSelect(int left, int right, int k) {
int pivotIndex;
Review comment:
it seems that we could reduce the scope of pivotIndex by moving it inside of
the do..while loop. I know it's used in the return statement, but we could
equally return left or right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11492) More Modern cloud dev script
[ https://issues.apache.org/jira/browse/SOLR-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960389#comment-16960389 ] Robert Bunch commented on SOLR-11492: - figured out what was going on w/ my fgc.sh script. ref the updated one in commentary in SOLR-13394. that's a variant of what I use to monitor the kajillion nodes I'm responsible for. > More Modern cloud dev script > > > Key: SOLR-11492 > URL: https://issues.apache.org/jira/browse/SOLR-11492 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 8.0 >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Minor > Attachments: SOLR-11492.patch, cloud.sh, cloud.sh, cloud.sh, > cloud.sh, cloud.sh, cloud.sh, cloud.sh > > > Most of the scripts in solr/cloud-dev do things like start using java -jar > and other similarly ancient techniques. I recently decided I really didn't > like that it was a pain to setup a cloud to test a patch/feature and that > often one winds up needing to blow away existing testing so working on more > than one thing at a time is irritating... so here's a script I wrote, if > folks like it I'd be happy for it to be included in solr/cloud-dev -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13394) Change default GC from CMS to G1
[
https://issues.apache.org/jira/browse/SOLR-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960387#comment-16960387
]
Robert Bunch edited comment on SOLR-13394 at 10/26/19 4:17 PM:
---
late chiming in on this... but I agree, it'd be very, very helpful for it to
be a knob that can be turned (like -debug). Perhaps also enable JMX w/ the
same switch?
I manage a lot of clusters, Solr and ES, and any means of gathering info –
specifically old gen FGC's – is invaluable. Discovering the JVMs in the first
place is critical.
{code:java}
#!/bin/bash
for pid in `pgrep -f start.jar`; do
# for pid in `pgrep java`; do # normally I use this, as I don't have a local
ZK...
st=`ps -eo pid,etime | grep ${pid} | awk '{$1=""; print $0}'`
cli=`${JAVA_HOME}/bin/jinfo $pid 2>/dev/null | grep -i xmx | awk '{print
$3}'` # placement of -Xmx will vary
if [[ "`uname`" == "Darwin" ]]; then
port=`lsof -p $pid | grep TCP | grep LISTEN | grep :89 | awk '{print $9}'`
else
port=`netstat -nap 2>/dev/null | grep $pid | grep :89 | grep LISTEN | awk
'{print $4}'` # this will vary, JMX port conflict?
fi
echo $port | grep ":89" >/dev/null; is_solr_port=$?
if [[ $is_solr_port -eq 0 ]]; then
fgc=`${JAVA_HOME}/bin/jstat -gcutil $pid | awk '{print $9, $10}' | tail
-1` # this will vary, java 7 vs 8 vs... get count, time
if [[ "$cli" == "" ]]; then cli="UNKNOWN"; fi
printf "%s %6d %'9d %'11.2f %s %s\n" $port $pid $fgc $cli $st
fi
done | sort | egrep --color=always '^.*?[0-9]{4} +[0-9]{3,} +[0-9,]{4,} .*|$' #
change throttle of redline at 3rd grouping
for j in $(seq 1 60); do echo -n '.'; sleep 60; done{code}
when it works (port, pid, # old gen FGCs, total old gen FGC time, -Xmx param,
process uptime (days-hours:min:sec)
{code:java}
$ ./fgc-jira.sh
*:8981 15685 0 0.00 -Xmx1g 06:15:13
*:8982 15779 0 0.00 -Xmx1g 06:15:09
*:8983 15873 0 0.00 -Xmx1g 06:15:04
*:8984 15967 0 0.00 -Xmx1g 06:15:00
...^C (it'll tick off a dot for each minute, then an hour later here's another
sample){code}
when I can't get to that info (in this case, :8985 is 8.2 running vanilla w/
-XX:+PerfDisableSharedMem)
{code:java}
$ ./fgc-jira.sh
sun.jvmstat.monitor.MonitorException: 17297 not found
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.(PerfDataBuffer.java:84)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.LocalMonitoredVm.(LocalMonitoredVm.java:68)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.MonitoredHostProvider.getMonitoredVm(MonitoredHostProvider.java:77)
at jdk.jcmd/sun.tools.jstat.Jstat.logSamples(Jstat.java:107)
at jdk.jcmd/sun.tools.jstat.Jstat.main(Jstat.java:70)
Caused by: java.lang.IllegalArgumentException: Could not map vmid to user Name
at java.base/jdk.internal.perf.Perf.attach(Native Method)
at java.base/jdk.internal.perf.Perf.attachImpl(Perf.java:270)
at java.base/jdk.internal.perf.Perf.attach(Perf.java:200)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.(PerfDataBuffer.java:64)
... 4 more
./fgc-jira.sh: line 15: printf: -XX:+UseG1GC: invalid number
./fgc-jira.sh: line 15: printf: 58:12: invalid number
*:8981 15685 0 0.00 -Xmx1g 06:15:13
*:8982 15779 0 0.00 -Xmx1g 06:15:09
*:8983 15873 0 0.00 -Xmx1g 06:15:04
*:8984 15967 0 0.00 -Xmx1g 06:15:00
*:8985 17297 0 0.00 (blew up trying to get jinfo for this node)
...^C{code}
was (Author: bunchr):
late chiming in on this... but I agree, it'd be very, very helpful for it to
be a knob that can be turned (like -debug). Perhaps also enable JMX w/ the
same switch?
I manage a lot of clusters, Solr and ES, and any means of gathering info –
specifically old gen FGC's – is invaluable. Discovering the JVMs in the first
place is critical.
{code:java}
#!/bin/bash
for pid in `pgrep -f start.jar`; do
# for pid in `pgrep java`; do # normally I use this, as I don't have a local
ZK...
st=`ps -eo pid,etime | grep ${pid} | awk '{$1=""; print $0}'`
cli=`${JAVA_HOME}/bin/jinfo $pid 2>/dev/null | grep -i xmx | awk '{print
$3}'` # placement of -Xmx will vary
if [[ "`uname`" == "Darwin" ]]; then
port=`lsof -p $pid | grep TCP | grep LISTEN | grep :89 | awk '{print $9}'`
else
port=`netstat -nap 2>/dev/null | grep $pid | grep :89 | grep LISTEN | awk
'{print $4}'` # this will vary, JMX port conflict?
fi
echo $port | grep ":89" >/dev/null; is_solr_port=$?
if [[ $is_solr_port -eq 0 ]]; then
fgc=`${JAVA_HOME}/bin/jstat -gcutil $pid | awk '{print $9, $10}' | tail
-1` # this will vary, java 7 vs 8 vs... get count, time
if [[ "$cli" == "" ]]; then cli="UNKNOWN"; fi
[jira] [Commented] (SOLR-13394) Change default GC from CMS to G1
[
https://issues.apache.org/jira/browse/SOLR-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960387#comment-16960387
]
Robert Bunch commented on SOLR-13394:
-
late chiming in on this... but I agree, it'd be very, very helpful for it to
be a knob that can be turned (like -debug). Perhaps also enable JMX w/ the
same switch?
I manage a lot of clusters, Solr and ES, and any means of gathering info –
specifically old gen FGC's – is invaluable. Discovering the JVMs in the first
place is critical.
{code:java}
#!/bin/bash
for pid in `pgrep -f start.jar`; do
# for pid in `pgrep java`; do # normally I use this, as I don't have a local
ZK...
st=`ps -eo pid,etime | grep ${pid} | awk '{$1=""; print $0}'`
cli=`${JAVA_HOME}/bin/jinfo $pid 2>/dev/null | grep -i xmx | awk '{print
$3}'` # placement of -Xmx will vary
if [[ "`uname`" == "Darwin" ]]; then
port=`lsof -p $pid | grep TCP | grep LISTEN | grep :89 | awk '{print $9}'`
else
port=`netstat -nap 2>/dev/null | grep $pid | grep :89 | grep LISTEN | awk
'{print $4}'` # this will vary, JMX port conflict?
fi
echo $port | grep ":89" >/dev/null; is_solr_port=$?
if [[ $is_solr_port -eq 0 ]]; then
fgc=`${JAVA_HOME}/bin/jstat -gcutil $pid | awk '{print $9, $10}' | tail
-1` # this will vary, java 7 vs 8 vs... get count, time
if [[ "$cli" == "" ]]; then cli="UNKNOWN"; fi
printf "%s %6d %'9d %'11.2f %s %s\n" $port $pid $fgc $cli $st
fi
done | sort | egrep --color=always '^.*?[0-9]{4} +[0-9]{3,} +[0-9,]{4,} .*|$' #
change throttle of redline at 3rd grouping
for j in $(seq 1 60); do echo -n '.'; sleep 60; done{code}
when it works (port, pid, # old gen FGCs, total old gen FGC time, -Xmx param,
process uptime (days-hours:min:sec)
{code:java}
$ ./fgc-jira.sh
*:8981 15685 0 0.00 -Xmx1g 06:15:13
*:8982 15779 0 0.00 -Xmx1g 06:15:09
*:8983 15873 0 0.00 -Xmx1g 06:15:04
*:8984 15967 0 0.00 -Xmx1g 06:15:00
...^C (it'll tick off a dot for each minute, then an hour later here's another
sample){code}
when I can't get to that info (in this case, :8985 is 8.2 running vanilla w/
-XX:+PerfDisableSharedMem)
{code:java}
$ ./fgc-jira.sh
sun.jvmstat.monitor.MonitorException: 17297 not found
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.(PerfDataBuffer.java:84)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.LocalMonitoredVm.(LocalMonitoredVm.java:68)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.MonitoredHostProvider.getMonitoredVm(MonitoredHostProvider.java:77)
at jdk.jcmd/sun.tools.jstat.Jstat.logSamples(Jstat.java:107)
at jdk.jcmd/sun.tools.jstat.Jstat.main(Jstat.java:70)
Caused by: java.lang.IllegalArgumentException: Could not map vmid to user Name
at java.base/jdk.internal.perf.Perf.attach(Native Method)
at java.base/jdk.internal.perf.Perf.attachImpl(Perf.java:270)
at java.base/jdk.internal.perf.Perf.attach(Perf.java:200)
at
jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.(PerfDataBuffer.java:64)
... 4 more
./fgc-jira.sh: line 15: printf: -XX:+UseG1GC: invalid number
./fgc-jira.sh: line 15: printf: 58:12: invalid number
$ ./fgc-jira.sh
*:8981 15685 0 0.00 -Xmx1g 06:15:13
*:8982 15779 0 0.00 -Xmx1g 06:15:09
*:8983 15873 0 0.00 -Xmx1g 06:15:04
*:8984 15967 0 0.00 -Xmx1g 06:15:00
*:8985 17297 0 0.00 (blew up trying to get jinfo for this node)
...^C{code}
> Change default GC from CMS to G1
>
>
> Key: SOLR-13394
> URL: https://issues.apache.org/jira/browse/SOLR-13394
> Project: Solr
> Issue Type: Improvement
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-13394.patch, SOLR-13394.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> CMS has been deprecated in new versions of Java
> (http://openjdk.java.net/jeps/291). This issue is to switch Solr default from
> CMS to G1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] pcsanwald removed a comment on issue #966: LUCENE-9024 Optimize IntroSelector to use median of medians
pcsanwald removed a comment on issue #966: LUCENE-9024 Optimize IntroSelector to use median of medians URL: https://github.com/apache/lucene-solr/pull/966#issuecomment-546330004 @jpountz thanks for the detailed review! I will be addressing your comments today. One comment: > it should be `to+random.nextInt(5)`, is it what you are proposing? That's part of it. but even with a right parameter of `to+0`, the code is never exercised over the whole array, as it's exclusive. So, I am not 100% sure I am right, but, it seems to me that there is no way to quickSelect on an entire array, as `to = arr.len` is still exclusive. I guess this has not been a problem in practice, but, I wanted to raise it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gus Heck reassigned SOLR-13749:
---
Assignee: Gus Heck
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr.search.join.XCJFQueryParserPlugin"}}{{>}}
> {{ }}{{<}}{{str}} {{name}}{{=}}{{"routerField"}}{{>vin}}
> {{}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"hash_range"}}
>
[jira] [Updated] (SOLR-13822) Isolated Classloading from packages
[
https://issues.apache.org/jira/browse/SOLR-13822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-13822:
--
Attachment: SOLR-13822.patch
> Isolated Classloading from packages
> ---
>
> Key: SOLR-13822
> URL: https://issues.apache.org/jira/browse/SOLR-13822
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-13822.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Design is here:
> [https://docs.google.com/document/d/15b3m3i3NFDKbhkhX_BN0MgvPGZaBj34TKNF2-UNC3U8/edit?ts=5d86a8ad#]
>
> main features:
> * A new file for packages definition (/packages.json) in ZK
> * Public APIs to edit/read the file
> * The APIs are registered at {{/api/cluster/package}}
> * Classes can be loaded from the package classloader using the
> {{:}} syntax
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
