[
https://issues.apache.org/jira/browse/SOLR-17007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774810#comment-17774810
]
Elia Porciani commented on SOLR-17007:
--------------------------------------
The problem shows up when there are multiple segments and one of them has not
vector values for the field requested in the function query.
Indeed, the tests fail when there are 2 segments in the index instead of 1.
{code:java}
@Override
public FunctionValues getValues(Map<Object, Object> context, LeafReaderContext
readerContext)
throws IOException {
final FloatVectorValues vectorValues =
readerContext.reader().getFloatVectorValues(fieldName);
if (vectorValues == null) {
throw new IllegalArgumentException(
"no float vector value is indexed for field '" + fieldName + "'");
}
....{code}
readerContext.reader().getFloatVectorValues(fieldName) returns null if there
are no floatVectorValues for that field name in that segment. This is
counterintuitive because in the same class, other methods the behavior is
different.
For instance:
{code:java}
getSortedDocValues(String field)
{code}
returns null if docValues are not defined for that field, but it returns an
empty list if there are no values in the segments. I think getFloatVectorValues
and getByteVectorValues should align with the same behavior of docvalues.
Unfortunately, the problem is in the Lucene code and it must be fixed there. In
the next weeks, I'm planning to open a PR for fixing this bug.
> TestDenseVectorFunctionQuery reproducible failures
> --------------------------------------------------
>
> Key: SOLR-17007
> URL: https://issues.apache.org/jira/browse/SOLR-17007
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Chris M. Hostetter
> Priority: Major
> Attachments: apache_solr_Solr-NightlyTests-main_928.log.txt,
> apache_solr_Solr-NightlyTests-main_931.log.txt,
> thetaphi_solr_Solr-main-Linux_14822.log.txt
>
>
> In the past week, the same 5 test methods of TestDenseVectorFunctionQuery
> have all failed 3 times - in the same 3 jenkins builds (ie: same master seed
> - which reproduces locally for me) and all of the test (method) failures have
> the same root cause ... strongly suggesting that some aspect of the static,
> or test class level, randomization is breaking these methods.
>
> Recent example...
> {noformat}
> ./gradlew test --tests TestDenseVectorFunctionQuery
> -Dtests.seed=749AD19AB618219E -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> -Dtests.locale=fr-MQ -Dtests.timezone=Asia/Novosibirsk -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> ...
> org.apache.solr.search.function.TestDenseVectorFunctionQuery >
> floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero FAILED
> java.lang.RuntimeException: Exception during query
> at
> __randomizedtesting.SeedInfo.seed([749AD19AB618219E:E0B29A3AECE5D888]:0)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:989)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:947)
> at
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero(TestDenseVectorFunctionQuery.java:173)
> ...
> Caused by:
> java.lang.IllegalArgumentException: no float vector value is indexed
> for field 'vector2'
> at
> org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource.getValues(FloatKnnVectorFieldSource.java:45)
> at
> org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:48)
> at
> org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
> at
> org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
> at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:135)
> at
> org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:515)
> at org.apache.lucene.search.Weight.bulkScorer(Weight.java:165)
> at
> org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:368)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:759)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:549)
> at
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1878)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1695)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
> at
> org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1696)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:422)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:467)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2901)
> at org.apache.solr.util.TestHarness.query(TestHarness.java:352)
> at org.apache.solr.util.TestHarness.query(TestHarness.java:332)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:961)
> ...
> > Task :solr:core:test FAILEDERROR: The following test(s) have failed:
> -
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero
> (:solr:core)
> Test output:
> /home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
> Reproduce with: gradlew :solr:core:test --tests
> "org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero"
> -Ptests.jvms=5 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
> -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
> -Ptests.seed=749AD19AB618219E -Ptests.multiplier=2 -Ptests.locale=fr-MQ
> -Ptests.timezone=Asia/Novosibirsk -Ptests.nightly=true
> -Ptests.file.encoding=UTF-8
> -Ptests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> -
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput
> (:solr:core)
> Test output:
> /home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
> Reproduce with: gradlew :solr:core:test --tests
> "org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput"
> -Ptests.jvms=5 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
> -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
> -Ptests.seed=749AD19AB618219E -Ptests.multiplier=2 -Ptests.locale=fr-MQ
> -Ptests.timezone=Asia/Novosibirsk -Ptests.nightly=true
> -Ptests.file.encoding=UTF-8
> -Ptests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> -
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity
> (:solr:core)
> Test output:
> /home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
> Reproduce with: gradlew :solr:core:test --tests
> "org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity"
> -Ptests.jvms=5 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
> -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
> -Ptests.seed=749AD19AB618219E -Ptests.multiplier=2 -Ptests.locale=fr-MQ
> -Ptests.timezone=Asia/Novosibirsk -Ptests.nightly=true
> -Ptests.file.encoding=UTF-8
> -Ptests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> -
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity
> (:solr:core)
> Test output:
> /home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
> Reproduce with: gradlew :solr:core:test --tests
> "org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity"
> -Ptests.jvms=5 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
> -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
> -Ptests.seed=749AD19AB618219E -Ptests.multiplier=2 -Ptests.locale=fr-MQ
> -Ptests.timezone=Asia/Novosibirsk -Ptests.nightly=true
> -Ptests.file.encoding=UTF-8
> -Ptests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> -
> org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero
> (:solr:core)
> Test output:
> /home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
> Reproduce with: gradlew :solr:core:test --tests
> "org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero"
> -Ptests.jvms=5 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
> -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
> -Ptests.seed=749AD19AB618219E -Ptests.multiplier=2 -Ptests.locale=fr-MQ
> -Ptests.timezone=Asia/Novosibirsk -Ptests.nightly=true
> -Ptests.file.encoding=UTF-8
> -Ptests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Solr/Solr-NightlyTests-main/test-data/enwiki.random.lines.txt
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
