Hi,
are your collections using stateFormat 1 or 2? In version 1 all state
was stored in one file while in version 2 each collection has its own
state.json. I assume that in the old version it could happen that the
common file still contains state for a collection that was deleted. So I
would read out the clusterstate.json in ZooKeeper and manually verify
its content.
regards,
Hendrik
On 11.03.2018 19:27, Atita Arora wrote:
Hi ,
I am working on an application which involves working on a highly
distributed Solr cloud environment. The application supports multi-tenancy
and we have around 250-300 collections on Solr where each client has their
own collection with a new shard being created as clientid-<timestamp_long>
where the timestamp is whenever the new data comes in for the client
(typically every 4-8 hrs) , the reason for this convention is to make sure
when the Indexes are being built (on demand) the timestamp matches closely
to the time when the last indexing was run (the earlier shard is
de-provisioned as soon as the new one is created). Whenever the indexing is
triggered it first makes a DB entry and then creates a catalog with
timestamp in solr.
The Solr cloud has 10 Nodes distributed geographically among 10 datacenters.
The replication factor is 2. The Solr version is 5.3.2.
Coming to my problem - I had to write a utility to ensure that the DB
insert timestamp matches closely to the Solr index timestamp wherein I can
ensure that if the difference between DB timestamp and Solr Index tinestamp
is <= 2 hrs , we have fresh index. The new index contains revised prices of
products or offers etc which are critical to be updated as in when they
come. Hence this utility is to track that the required updates have been
successfully made.
I used *CLUSTERSTATUS* api for this task. It is serving the purpose well so
far , but pretty recently our solr cloud started complaining of strange
things because of which the *CLUSTERSTATUS* api keeps returning as error.
The error claims to be of missing config & sometime missing collections
like.
org.apache.solr.common.SolrException: Could not find collection :
1785-1520548816454
org.apache.solr.common.SolrException: Could not find collection :
1785-1520548816454
at
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:165)
at
org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(ClusterStatus.java:110)
at
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$19.call(CollectionsHandler.java:614)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:166)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:678)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
The other times it would complain of missing the config for same or
different client id- timestamp like :
1532-1518669619526_shard1_replica3:
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
Specified config does not exist in ZooKeeper:1532-1518669619526I
I would really appreciate if :
1. Someone can possibly guide me as to whats going on Solr Cloud
2. If CLUSTERSTATUS is the right pick to build such utility. Do we
have any other option?
Thanks for any pointers and suggestions.
Appreciate your attention looking this through.
Atita
