Good question, what do the logs say? You’ve provided very little information
to help diagnose the issue.
As to your observation that atomic updates are expensive, that’s true. Under
the covers, Solr has to go out and fetch the document, overlay your changes
and then re-index the full document. So, indeed, it actually takes more work
as far as Solr is concerned than just having the entire document re-sent by
the client.
I don’t know offhand what the root cause of the difference in ingestion rates
is…
Best,
Erick
> On Aug 10, 2020, at 10:23 AM, Anshuman Singh <singhanshuma...@gmail.com>
> wrote:
>
> Hi,
>
> We have a SolrCloud cluster with 10 nodes. We have 6B records ingested in
> the Collection. Our use case requires atomic updates ("inc") on 5 fields.
> Now almost 90% documents are atomic updates and as soon as we start our
> ingestion pipelines, multiple shards start going into recovery, sometimes
> all replicas of some shards go into down state.
> The ingestion rate is also too slow with atomic updates, 4-5k per second.
> We were able to ingest records without atomic updates at the rate of 50k
> records per second without any issues.
>
> What I'm suspecting is, the fact that these "inc" atomic updates
> require fetching of fields before indexing can cause slow rates but what
> I'm not getting is, why are the replicas going into recovery?
>
> Regards,
> Anshuman
