[
https://issues.apache.org/jira/browse/SOLR-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17349950#comment-17349950
]
xuanyu huang commented on SOLR-15417:
-------------------------------------
Hi Takashi,
Maybe wait for a bit more longer, 20s, my query returns 2 records.
This is my code:
{code:java}
solrClient = new
ConcurrentUpdateSolrClient.Builder(solrHome).withThreadCount(4).withQueueSize(500).build();
@Override
public void indexFromMap(List<Map<String, Object>> maps) throws IOException,
SolrServerException {
List<SolrInputDocument> batch = new ArrayList<>();
int idx = 1;
for (Map<String, Object> map : maps) {
if (map.containsKey("record_uuid")) {
SolrInputDocument doc = new SolrInputDocument();
if (idx == 3) {
doc.addField("id", map.get("record_uuid") + "_invalid");
} else {
doc.addField("id", map.get("record_uuid"));
}
idx++;
doc.addField("hasUserAssertions", new HashMap<String, Object>() {{
put("set", true); }});
// this makes sure update only succeeds when record with specified id
exists
doc.addField("_version_", 1);
// update schema if needed, this line create the field in schema
syncDocFieldsWithSOLR(doc);
logger.debug("Added solr doc for record: -------- " + doc.get("id"));
batch.add(doc);
}
}
updateBatch(batch);
}
private void updateBatch(List<SolrInputDocument> batch) {
if (!batch.isEmpty()) {
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.setAction(UpdateRequest.ACTION.COMMIT, false, false);
updateRequest.setParam("failOnVersionConflicts", "false");
updateRequest.add(batch);
updateRequest.lastDocInBatch();
try {
UpdateResponse process = updateRequest.process(solrClient);
logger.debug("xhk205 process = " + process.toString());
} catch (Exception e) {
logger.error("Failed to update solr doc, error message: " +
e.getMessage(), e);
}
}
}{code}
Below is the log
{code:java}
2021-05-23 17:59:14,967 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
-------- id=31fbf6f3-df72-41ae-a1f0-be5066122c3d
2021-05-23 17:59:14,967 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
-------- id=c0ee1a86-1df6-40b2-950c-bdde40b1c46e
2021-05-23 17:59:14,967 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
-------- id=552f7f85-cbc7-44b4-b7ba-0aa4ddc1e863_invalid
2021-05-23 17:59:14,967 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
-------- id=af56ce03-e664-421a-85ac-fbb839bbb140
2021-05-23 17:59:14,967 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
-------- id=6bdc3c1d-d21a-43e3-aa84-baeeda601bb3
2021-05-23 17:59:23,603 DEBUG: [SolrIndexDAOImpl] - xhk205 process = {NOTE=the
request is processed in a background stream}
2021-05-23 17:59:23,878 ERROR: [ConcurrentUpdateSolrClient] -
errororg.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://localhost:8983/solr/biocache: Conflict
request:
http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2Remote
error message: Document not found for update.
id=552f7f85-cbc7-44b4-b7ba-0aa4ddc1e863_invalid at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:218)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
See the {color:#de350b}*Document not found for update. id=*
{color}{color:#172b4d}it's in DistributedUpdateProcessor.java{color}
{color:#172b4d}My query is
[http://localhost:8983/solr/biocache/select?fq=hasUserAssertions:true&q=*:*]{color}
{color:#172b4d}"responseHeader": {"zkConnected": true,"status": 0,"QTime":
2,"params": {"q": "*:*","fq": "hasUserAssertions:true"}},"response":
{"numFound": 2,"start": 0,"docs": [{color}
> exception in updateRequest caused all subsequent update fail
> ------------------------------------------------------------
>
> Key: SOLR-15417
> URL: https://issues.apache.org/jira/browse/SOLR-15417
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Affects Versions: 8.5.1
> Reporter: xuanyu huang
> Priority: Minor
>
> Hi there,
> I'm using solrj 8.8.2 for a 8.5.1 solr server. I have a list of records and
> in a for loop I construct an updateRequest to update each record.
> Code looks like this
> {code:java}
> for (Map<String, Object> map : maps) {
> if (map.containsKey("record_uuid")) {
> UpdateRequest updateRequest = new UpdateRequest();
> updateRequest.setAction( UpdateRequest.ACTION.COMMIT, false, false);
> SolrInputDocument doc = new SolrInputDocument();
> if (idx == 3) {
> doc.addField("id", map.get("record_uuid") + "_invalid");
> } else {
> doc.addField("id", map.get("record_uuid"));
> }
> idx++;
> doc.addField("hasUserAssertions", new HashMap<String, Object>() {{
> put("set", true); }});
> // this makes sure update only succeeds when record with specified id
> exists
> doc.addField("_version_", 1);
> logger.debug("Added solr doc for record: " + doc.get("id"));
> updateRequest.add(doc);
> try {
> updateRequest.setParam("failOnVersionConflicts", "false");
> UpdateResponse process = updateRequest.process(solrClient);
> System.out.println("xhk205 process = " + process.toString());
> } catch (Exception e) {
> logger.error("Failed to update solr doc, error message: " +
> e.getMessage(), e);
> }
> }{code}
> There are 5 requests in total and I intentionally set the id in 3rd request
> to be an invalid id so that updateRequet for 3rd record should fail. (This is
> to mimic the situation where the record to be updated no longer exists in
> solr, so I only want those updates with a valid id to succeed, those updates
> with an invalid id should fail/rejected instead of creating a new reocrd in
> solr, so I used __version__=1).
>
> Also I used the syntax to do partial update.
> The variable doc looks like this
> {code:java}
> {
> "id":"2d4b625d-8809-461f-b19b-d0c963e038ed",
> "hasUserAssertions":{"set":true}
> }
> {code}
>
> {color:#de350b}Since each update is put into its own request, I suppose only
> the 3rd request will fail because there's no record with that id and I've set
> __version__{color} {color:#de350b}to 1. But the reality is, only the first 2
> records were updated and other 3 not.{color}
> {color:#de350b}When I queried in solr admin console after the update, with
> [http://localhost:8983/solr/biocache/select?fq=hasUserAssertions:true&q=*:*]
> there were only 2 records returned instead of 4.{color}
>
> Below is the log of IntelliJ IDEA:
>
> {code:java}
> - Added solr doc for record: id=429cfa88-2e18-46b0-ab9f-f4efd9e36c3c
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=5a80561b-a68d-46a3-a59b-03d267f35d0e
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=baf7af1f-1525-403a-95bf-e28e432f1b12
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=4ea76605-c262-409b-845e-213f11ea4e34
> xhk205 process = {NOTE=the request is processed in a background stream}{code}
> {code:java}
> 2021-05-19 14:12:16,827 ERROR: [ConcurrentUpdateSolrClient] - error
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://localhost:8983/solr/biocache: Conflict request:
> http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2
> Remote error message: Document not found for update.
> id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid at
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394)
> at
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0{code}
>
>
> {color:#de350b}The 3rd update obviously caused an exception. But why 4th and
> 5th updates didn't succeed? Is it possible that this exception caused solr
> client or server in some non-useable state so all subsequent updates
> failed?{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
