Hi there,
We analyzed the source code of Apache Solr and found a number of
issues that we would like to report. We configured Solr using the
films collection from the quick start tutorial [2]. For every issue
that we found we can provide a request URL for which Solr returns an
HTTP 500 error (usually due to an uncaught exception).
Please find below two example issues we found, together with an
explanation of the cause and a patch fixing the bug (attached patches
pass the unit tests). The issues we would like to report are similar
to those. We found them with help from Diffblue Microservice Testing
[1].
We would like to know:
* Should we open JIRA tickets for each one of them?
* We can provide a URL that triggers the problem, a stack trace, and
information about the server setup. What other information would you
like to see?
I look forward to your response.
Best regards,
César
PS. If this question is not suitable for this list, please indicate
where to follow up.
=== Issue 1: Null pointer exception due to unhandled case in
ComplexPhraseQueryParser ===
Assume that we request the following URL:
/solr/films/select?q={!complexphrase}genre:"-om*"
Handling this query involves constructing a SpanQuery, which happens
in the rewrite method of ComplexPhraseQueryParser. In particular, the
expression is decomposed into a BooleanQuery, which has exactly one
clause, namely the negative clause -genre:”om*”. The rewrite method
then further transforms this into a SpanQuery; in this case, it goes
into the path that handles complex queries with both positive and
negative clauses. It extracts the subset of positive clauses - note
that this set of clauses is empty for this query. The positive clauses
are then combined into a SpanNearQuery (around line 340), which is
then used to build a SpanNotQuery.
Further down the line, the field attribute of the SpanNearQuery is
accessed and used as an index into a TreeMap. But since we had an
empty set of positive clauses, the SpanNearQuery does not have its
field attribute set, so we get a null here - this leads to an
exception.
A possible fix would be to detect the situation where we have an empty
set of positive clauses and include a single synthetic clause that
matches either everything or nothing. See attached file
0001-Fix-NullPointerException.patch.
=== Issue 2: StringIndexOutOfBoundsException when expanding macros ===
Assume that we request the following URL:
/solr/films/select?a=${${b}}
Parameter macro expansion [3] seems to take place in
org.apache.solr.request.macro.MacroExpander._expand(String val).
However, this method throws a StringIndexOutOfBoundsException for the
URL above. From reading the code it seems that macros are not expanded
inside curly brackets ${...}, and so the “${b}” inside “${${b}}”
should not be expanded. But the function seems to fail to detect this
specific case and graciously refuse to expand it. Instead, it
unexpectedly throws the exception. A possible fix could be updating
the ‘idx’ variable when the StrParser detects that no valid identifier
can be found inside the brackets. See attached file
0001-Macro-expander-fail-gracefully-on-unsupported-syntax.patch.
References:
[1] https://www.diffblue.com/labs/
[2] https://lucene.apache.org/solr/guide/7_6/solr-tutorial.html
[3] http://yonik.com/solr-query-parameter-substitution/
--
Diffblue Limited, a company registered in England and Wales number
09958102, with its registered office at Ramsey House, 10 St. Ebbes Street,
Oxford, OX1 1PT, England
From 5cf5371c1d6febf1a6423aa65aecde8be7eed77c Mon Sep 17 00:00:00 2001
From: Johannes Kloos <johannes.kl...@diffblue.com>
Date: Thu, 24 Jan 2019 16:54:20 +0000
Subject: [PATCH] Fix NullPointerException.
---
.../src/java/org/apache/lucene/search/spans/SpanNearQuery.java | 2 ++
.../lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.java | 7 ++++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/lucene/core/src/java/org/apache/lucene/search/spans/SpanNearQuery.java b/lucene/core/src/java/org/apache/lucene/search/spans/SpanNearQuery.java
index 17b9e51..a312ad2 100644
--- a/lucene/core/src/java/org/apache/lucene/search/spans/SpanNearQuery.java
+++ b/lucene/core/src/java/org/apache/lucene/search/spans/SpanNearQuery.java
@@ -130,6 +130,8 @@ public class SpanNearQuery extends SpanQuery implements Cloneable {
* @param inOrder true if order is important
*/
public SpanNearQuery(SpanQuery[] clausesIn, int slop, boolean inOrder) {
+ if (clausesIn.length == 0)
+ throw new IllegalArgumentException("SpanNearQuery with no clauses");
this.clauses = new ArrayList<>(clausesIn.length);
for (SpanQuery clause : clausesIn) {
if (this.field == null) { // check field
diff --git a/lucene/queryparser/src/java/org/apache/lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.java b/lucene/queryparser/src/java/org/apache/lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.java
index ffe0066..574589f 100644
--- a/lucene/queryparser/src/java/org/apache/lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.java
+++ b/lucene/queryparser/src/java/org/apache/lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.java
@@ -344,11 +344,16 @@ public class ComplexPhraseQueryParser extends QueryParser {
.toArray(new SpanQuery[positiveClauses.size()]);
SpanQuery include = null;
- if (includeClauses.length == 1) {
+ if (includeClauses.length == 0) {
+ // Invent a positive clause out of thin air.
+ include = new SpanTermQuery(new Term(field,
+ "Dummy clause because no terms found - must match nothing"));
+ } else if (includeClauses.length == 1) {
include = includeClauses[0]; // only one positive clause
} else {
// need to increase slop factor based on gaps introduced by
// negatives
+ assert (includeClauses.length >= 2);
include = new SpanNearQuery(includeClauses, slopFactor + numNegatives,
inOrder);
}
--
2.7.4
From aba36be6fd355f29d0967fd9e2e1f6baf326bb9e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9sar=20Rodr=C3=ADguez?= <cesar.rodrig...@diffblue.com>
Date: Thu, 24 Jan 2019 16:58:44 +0000
Subject: [PATCH] Macro expander: fail gracefully on unsupported syntax
---
solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java | 1 +
1 file changed, 1 insertion(+)
diff --git a/solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java b/solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java
index 9d432fa..19999b5 100644
--- a/solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java
+++ b/solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java
@@ -182,6 +182,7 @@ public class MacroExpander {
} catch (SyntaxError syntaxError) {
// append the part we would have skipped
sb.append( val.substring(matchedStart, start) );
+ idx = start;
continue;
}
--
2.7.4
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
