[
https://issues.apache.org/jira/browse/SOLR-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Angel Todorov updated SOLR-11139:
---------------------------------
Description:
If an exception is thrown below, it will exit the while loop, therefore
stopping building of the suggester index. In fact, it will not build anything,
because the exception is not handled in any way. This is highly unacceptable,
because in an enterprise environment there will always be data that does not
conform to the expected rules.
while (true) {
BytesRef term = termsEnum.next();
if (term == null) {
break;
}
int ngramCount = countGrams(term);
if (ngramCount > grams) {
throw new IllegalArgumentException("tokens must not contain separator
byte; got token=" + term + " but gramCount=" + ngramCount + ", which is greater
than expected max ngram size=" + grams);
}
if (ngramCount == 1) {
totTokens += termsEnum.totalTermFreq();
}
builder.add(Util.toIntsRef(term, scratchInts),
encodeWeight(termsEnum.totalTermFreq()));
}
was:
If an exception is thrown below, it will exit the while loop, therefore
stopping building of the suggester index. This is highly unacceptable, because
in an enterprise environment there will always be data that does not conform to
the expected rules.
while (true) {
BytesRef term = termsEnum.next();
if (term == null) {
break;
}
int ngramCount = countGrams(term);
if (ngramCount > grams) {
throw new IllegalArgumentException("tokens must not contain separator
byte; got token=" + term + " but gramCount=" + ngramCount + ", which is greater
than expected max ngram size=" + grams);
}
if (ngramCount == 1) {
totTokens += termsEnum.totalTermFreq();
}
builder.add(Util.toIntsRef(term, scratchInts),
encodeWeight(termsEnum.totalTermFreq()));
}
> FreeTextSuggester exits on first error while building suggester index
> ---------------------------------------------------------------------
>
> Key: SOLR-11139
> URL: https://issues.apache.org/jira/browse/SOLR-11139
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Suggester
> Affects Versions: 5.5.4
> Reporter: Angel Todorov
> Priority: Blocker
>
> If an exception is thrown below, it will exit the while loop, therefore
> stopping building of the suggester index. In fact, it will not build
> anything, because the exception is not handled in any way. This is highly
> unacceptable, because in an enterprise environment there will always be data
> that does not conform to the expected rules.
> while (true) {
> BytesRef term = termsEnum.next();
> if (term == null) {
> break;
> }
> int ngramCount = countGrams(term);
> if (ngramCount > grams) {
> throw new IllegalArgumentException("tokens must not contain
> separator byte; got token=" + term + " but gramCount=" + ngramCount + ",
> which is greater than expected max ngram size=" + grams);
> }
> if (ngramCount == 1) {
> totTokens += termsEnum.totalTermFreq();
> }
> builder.add(Util.toIntsRef(term, scratchInts),
> encodeWeight(termsEnum.totalTermFreq()));
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
