Amrit Sarkar created SOLR-18591:
-----------------------------------
Summary: Tolerant distributed search double-counts shard cpuTime
on partial failure
Key: SOLR-18591
URL: https://issues.apache.org/jira/browse/SOLR-18591
Project: Solr
Issue Type: Bug
Reporter: Amrit Sarkar
With {{solr.log.cputime=true}} and {{shards.tolerant=true}},
{{SearchHandler.processComponentsDistrib}} adds a {{ShardRequest}}'s shard CPU
to {{totalShardCpuTime}} [in the partial-failure
branch|https://github.com/apache/solr/blob/0d6b2e32f59/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L670-L674]
and then [again at the bottom of the response
loop|https://github.com/apache/solr/blob/0d6b2e32f59/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L696-L698].
So when one shard fails but others succeed, the final {{cpuTime}} over-reports
by that request's shard CPU. The response header and the log also end up with
several {{cpuTime}} entries, and since {{NamedList.get}} returns the first one,
clients see a partial total.
With two shards reporting a fixed 1000 ms each and one of them throwing:
* {{distrib.singlePass=true}}: header {{cpuTime}} is {{[1000, 2000]}}, expected
{{[1000]}}
* two-phase: header {{cpuTime}} is {{[1000, 3000]}}, expected {{[2000]}}
The partial-branch block came in with
[SOLR-17151|https://issues.apache.org/jira/browse/SOLR-17151]. Removing it
fixes both cases, and the existing CPU and query-limits tests still pass. Same
code on {{branch_10x}}.
Separately, {{shortCircuitedResults}} never writes the accumulated shard
{{cpuTime}}, so any query-limit short-circuit in a distributed search reports
only coordinator CPU.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
