[
https://issues.apache.org/jira/browse/SOLR-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433363#comment-17433363
]
Ishan Chattopadhyaya edited comment on SOLR-15705 at 10/24/21, 6:49 AM:
------------------------------------------------------------------------
I tried to reproduce some of these scenarios, and here's what I did:
h2. Started Solr 8.10
{code:java}
docker network create solr-network
# In one terminal window
docker run -it -h solr1 --name solr1 --net solr-network -p 18983:8983 solr:8.10
/opt/solr/bin/solr -c -f
# In another terminal window
docker run -it -h solr2 --name solr2 --net solr-network -p 28983:8983 solr:8.10
/opt/solr/bin/solr -c -f -z solr1:9983
{code}
h2. Scenario 1: Implicit routing
1. Created a collection with implicit router
{code:java}
curl --request GET \
--url
'http://localhost:18983/solr/admin/collections?action=CREATE&name=abc&numShards=2&router.name=implicit&shards=shard1%2Cshard2'
{code}
2. Added a document to shard1:
{code:java}
curl --request POST \
--url 'http://localhost:18983/solr/abc/update/json/docs?_route_=shard1' \
--header 'Content-Type: application/json' \
--data '{
"id": "india1",
"title_t": "hello how are you"
}'
{code}
3. Added a second document to shard2 & performed a commit
{code:java}
curl --request POST \
--url
'http://localhost:18983/solr/abc/update/json/docs?_route_=shard2&commit=true' \
--header 'Content-Type: application/json' \
--data '{
"id": "usa1",
"title_t": "i am fine"
}'
{code}
4. Sent a DBI for usa1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>usa1</id>
</delete>'
{code}
5. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, shard1 was placed on another node, so these requests were handled by
shard2. This document was successfully deleted.
5. Sent a DBI for india1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>india1</id>
</delete>'
{code}
6. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, we see that india1 was not deleted, even though the DBI query returned a
0 status (successful).
h2. Scenario 2: CompositeId routing
1. Create a collection
{code:java}
curl --request GET \
--url
'http://localhost:18983/solr/admin/collections?action=CREATE&name=com1&numShards=2'
{code}
2. Added two documents:
{code:java}
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs \
--header 'Content-Type: application/json' \
--data '{
"id": "india!1",
"title_t": "hello how are you"
}'
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs&commit=true \
--header 'Content-Type: application/json' \
--data '{
"id": "russia!1",
"title_t": "hi"
}'
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs&commit=true \
--header 'Content-Type: application/json' \
--data '{
"id": "usa!1",
"title_t": "hi"
}'
{code}
Here, {{usa!1}} goes to shard2, {{india!1}} and {{russia!1}} go to shard1. In
my case, shard1 was on 18983, shard2 was on 28983.
3. Deleted usa:1
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/com1/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>usa:1</id>
</delete>'
{code}
4. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/com1/select?q=*%3A*&shards=shard2'
{code}
Here, Solr was able to delete the document successfully.
I repeated the step 3 & 4 with "russia:1" and that document was also deleted
successfully.
h2. Conclusion
1. I can see that in implicit router scenario, the bug exists. However, looking
at the code [0], it seemed to me that the compositeIdRouter situation was
addressed, but not the implicit router.
2. Judging by the scenario 2 experiment I did (and please point out what I may
have missed), this problem doesn't exist for compositeId router.
[0] -
[https://github.com/apache/solr/pull/288/files#diff-dc18a65054da5e402a69909f9ee134b56dc496200d64dc8293e941d14d9961bdR324-R331]
[~makosten], can you please suggest what I could be missing? Thanks!
was (Author: ichattopadhyaya):
I tried to reproduce some of these scenarios, and here's what I did:
h2. Started Solr 8.10
{code}
docker network create solr-network
# In one terminal window
docker run -it -h solr1 --name solr1 --net solr-network -p 18983:8983 solr:8.10
/opt/solr/bin/solr -c -f
# In another terminal window
docker run -it -h solr2 --name solr2 --net solr-network -p 28983:8983 solr:8.10
/opt/solr/bin/solr -c -f -z solr1:9983
{code}
h2. Scenario 1: Implicit routing
1. Created a collection with implicit router
{code:java}
curl --request GET \
--url
'http://localhost:18983/solr/admin/collections?action=CREATE&name=abc&numShards=2&router.name=implicit&shards=shard1%2Cshard2'
{code}
2. Added a document to shard1:
{code:java}
curl --request POST \
--url 'http://localhost:18983/solr/abc/update/json/docs?_route_=shard1' \
--header 'Content-Type: application/json' \
--data '{
"id": "india1",
"title_t": "hello how are you"
}'
{code}
3. Added a second document to shard2 & performed a commit
{code:java}
curl --request POST \
--url
'http://localhost:18983/solr/abc/update/json/docs?_route_=shard2&commit=true' \
--header 'Content-Type: application/json' \
--data '{
"id": "usa1",
"title_t": "i am fine"
}'
{code}
4. Sent a DBI for usa1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>usa1</id>
</delete>'
{code}
5. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, shard1 was placed on another node, so these requests were handled by
shard2. This document was successfully deleted.
5. Sent a DBI for india1 to shard2
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>india1</id>
</delete>'
{code}
6. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/abc/select?q=*%3A*'
{code}
Here, we see that india1 was not deleted, even though the DBI query returned a
0 status (successful).
h2. Scenario 2: Composite routing
1. Create a collection
{code:java}
curl --request GET \
--url
'http://localhost:18983/solr/admin/collections?action=CREATE&name=com1&numShards=2'
{code}
2. Added two documents:
{code:java}
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs \
--header 'Content-Type: application/json' \
--data '{
"id": "india:1",
"title_t": "hello how are you"
}'
curl --request POST \
--url http://localhost:18983/solr/com1/update/json/docs&commit=true \
--header 'Content-Type: application/json' \
--data '{
"id": "russia:1",
"title_t": "hi"
}'
{code}
3. Deleted usa:1
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/com1/update?commit=true' \
--header 'Content-Type: application/xml' \
--data '<delete>
<id>india:1</id>
</delete>'
{code}
4. Searched for all documents:
{code:java}
curl --request GET \
--url 'http://localhost:18983/solr/com1/select?q=*%3A*&shards=shard2'
{code}
Here, Solr was able to delete the document successfully.
I repeated the step 3 & 4 with "russia:1" and that document was also deleted
successfully.
h2. Conclusion
1. I can see that in implicit router scenario, the bug exists. However, looking
at the code [0], it seemed to me that the compositeIdRouter situation was
addressed, but not the implicit router.
2. Judging by the scenario 2 experiment I did (and please point out what I may
have missed), this problem doesn't exist for compositeId router.
[0] -
[https://github.com/apache/solr/pull/288/files#diff-dc18a65054da5e402a69909f9ee134b56dc496200d64dc8293e941d14d9961bdR324-R331]
[~makosten], can you please suggest what I could be missing? Thanks!
> Distribute a DeleteById to all shards when using the CompositeId router with
> a router field defined and field value is missing in request
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-15705
> URL: https://issues.apache.org/jira/browse/SOLR-15705
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Michael Kosten
> Assignee: David Eric Pugh
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When issuing a DeleteById command for a collection using the CompositeId
> router with a router field defined and the route value is missing from the
> request, the DeleteById fails silently, even if it happens to have been sent
> to the correct shard. Instead of failing silently, the request could be
> forwarded to all shard leaders and from there to all replicas. Another
> required change is that the deletion would need to be performed even though
> the route value is missing. The deletion would be a no-op on the shards where
> the document does not exist.
> One use case for this feature is when the routing key value is volatile and
> you need to guarantee no duplicates exist when updating a document if it
> happens to change shards. Another use case is if you want to delete a
> document without referencing a database to retrieve the route field value.
> A work-around is to use a DeleteByQuery. However, my testing has found that
> DeleteById performs significantly better, even when sent to all shards.
> Additionally, I've found that a heavy mixed load of DeleteByQuery commands
> and Add commands can lead to node failures when there are multiple replicas.
> Additional comments are in SOLR-6910, however, that JIRA is specific to the
> implicit router and I mistakenly added them there.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
