sarkaramrit1993 commented on code in PR #5075:
URL: https://github.com/apache/solr/pull/5075#discussion_r4558360071
##########
changelog/unreleased/SOLR-5939.yml:
##########
@@ -0,0 +1,7 @@
+title: Distributed update errors from a merged stream are now recorded against
each request in the stream instead of the first request sent to the node
Review Comment:
Worth one sentence here: a 503 can arrive after the leader already applied
earlier updates in the stream. Those now get resent, so forwarded
non-idempotent atomic updates like `inc` can be applied twice.
##########
solr/core/src/java/org/apache/solr/update/StreamingSolrClients.java:
##########
@@ -127,26 +137,49 @@ class ErrorReportingConcurrentUpdateSolrClient extends
ConcurrentUpdateJettySolr
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
private final SolrCmdDistributor.Req req;
private final List<SolrError> errors;
+ private final Map<UpdateRequest, SolrCmdDistributor.Req> reqByUpdateRequest;
public ErrorReportingConcurrentUpdateSolrClient(Builder builder) {
super(builder);
this.req = builder.req;
this.errors = builder.errors;
+ this.reqByUpdateRequest = builder.reqByUpdateRequest;
}
@Override
public void handleError(Throwable ex) {
- log.error("Error when calling {} to {}", req, req.node.getUrl(), ex);
+ reqByUpdateRequest.remove(req.uReq);
+ recordError(ex, req);
+ }
+
+ @Override
+ protected void handleError(
+ Throwable ex, List<UpdateRequest> requests, List<String> failedIds,
String collection) {
+ if (requests.isEmpty()) {
+ handleError(ex);
+ return;
+ }
+ // The failure belongs to the stream as a whole, so record it once against
each request
+ // that was actually in the stream, instead of once against the request
this client
+ // happened to be built with.
+ for (UpdateRequest updateRequest : requests) {
+ SolrCmdDistributor.Req streamReq =
reqByUpdateRequest.remove(updateRequest);
+ recordError(ex, streamReq != null ? streamReq : req);
+ }
+ }
+
+ private void recordError(Throwable ex, SolrCmdDistributor.Req errorReq) {
+ log.error("Error when calling {} to {}", errorReq, errorReq.node.getUrl(),
ex);
Review Comment:
This now runs once per request in a failed stream, each with the full stack
trace, and `SolrCmdDistributor.doRetriesIfNeeded` logs another one per resubmit
since it passes `err.e`. So N requests in one failed stream give 2N identical
stack traces. Logging the trace once in `handleError(Throwable, List, List,
String)` and a line per request without it keeps the useful part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]