subject:"CLUSTERSTATUS API and Error loading specified collection \/ config in Solr 5.3.2."

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

2018-03-13 Thread Atita Arora

Hi Hendrik and Shalin,

Really appreciate your valuable inputs on this.

I looked up to the two issues that were being referred to (SOLR-8804 and
SOLR-10720) and thats exactly what I'm running into.
Glad they have been fixed in later versions.

Thanks much ,
Atita

On Tue, Mar 13, 2018 at 10:38 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> I think you are running into race conditions in the API which have been
> fixed. See SOLR-8804 and SOLR-10720. The first is available in 5.5.1 but
> the latter fix will be released in the upcoming 7.3 release. The best
> workaround for your version is to just retry a few times until the API
> succeeds.
>
> On Sun, Mar 11, 2018 at 11:57 PM, Atita Arora 
> wrote:
>
> > Hi ,
> >
> > I am working on an application which involves working on a highly
> > distributed Solr cloud environment. The application supports
> multi-tenancy
> > and we have around 250-300 collections on Solr where each client has
> their
> > own collection with a new shard being created as
> clientid-
> > where the timestamp is whenever the new data comes in for the client
> > (typically every 4-8 hrs) , the reason for this convention is to make
> sure
> > when the Indexes are being built (on demand) the timestamp matches
> closely
> > to the time when the last indexing was run (the earlier shard is
> > de-provisioned as soon as the new one is created). Whenever the indexing
> is
> > triggered it first makes a DB entry and then creates a catalog with
> > timestamp in solr.
> > The Solr cloud has 10 Nodes distributed geographically among 10
> > datacenters.
> > The replication factor is 2. The Solr version is 5.3.2.
> > Coming to my problem - I had to write a utility to ensure that the DB
> > insert timestamp matches closely to the Solr index timestamp wherein I
> can
> > ensure that if the difference between DB timestamp and Solr Index
> tinestamp
> > is <= 2 hrs , we have fresh index. The new index contains revised prices
> of
> > products or offers etc which are critical to be updated as in when they
> > come. Hence this utility is to track that the required updates have been
> > successfully made.
> > I used *CLUSTERSTATUS* api for this task. It is serving the purpose well
> so
> > far , but pretty recently our solr cloud started complaining of strange
> > things because of which the *CLUSTERSTATUS* api keeps returning as error.
> >
> > The error claims to be of missing config & sometime missing collections
> > like.
> >
> > org.apache.solr.common.SolrException: Could not find collection :
> > > 1785-1520548816454
> >
> > org.apache.solr.common.SolrException: Could not find collection :
> > 1785-1520548816454
> > at
> > org.apache.solr.common.cloud.ClusterState.getCollection(
> > ClusterState.java:165)
> > at
> > org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(
> > ClusterStatus.java:110)
> > at
> > org.apache.solr.handler.admin.CollectionsHandler$
> > CollectionOperation$19.call(CollectionsHandler.java:614)
> > at
> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> > CollectionsHandler.java:166)
> > at
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> > RequestHandlerBase.java:143)
> > at
> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> > HttpSolrCall.java:678)
> > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:215)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:179)
> > at
> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> > doFilter(ServletHandler.java:1652)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:585)
> >
> > The other times it would complain of missing the config for same or
> > different client id- timestamp like :
> >
> > 1532-1518669619526_shard1_replica3:
> > org.apache.solr.common.cloud.ZooKeeperException:org.apache.
> > solr.common.cloud.ZooKeeperException:
> > Specified config does not exist in ZooKeeper:1532-1518669619526I
> >
> > I would really appreciate if :
> >
> >
> >1. Someone can possibly guide me as to whats going on Solr Cloud
> >2. If CLUSTERSTATUS is the right pick to build such utility. Do we
> > have any other option?
> >
> >
> > Thanks for any pointers and suggestions.
> >
> > Appreciate your attention looking this through.
> >
> > Atita
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

2018-03-12 Thread Shalin Shekhar Mangar

I think you are running into race conditions in the API which have been
fixed. See SOLR-8804 and SOLR-10720. The first is available in 5.5.1 but
the latter fix will be released in the upcoming 7.3 release. The best
workaround for your version is to just retry a few times until the API
succeeds.

On Sun, Mar 11, 2018 at 11:57 PM, Atita Arora  wrote:

> Hi ,
>
> I am working on an application which involves working on a highly
> distributed Solr cloud environment. The application supports multi-tenancy
> and we have around 250-300 collections on Solr where each client has their
> own collection with a new shard being created as clientid-
> where the timestamp is whenever the new data comes in for the client
> (typically every 4-8 hrs) , the reason for this convention is to make sure
> when the Indexes are being built (on demand) the timestamp matches closely
> to the time when the last indexing was run (the earlier shard is
> de-provisioned as soon as the new one is created). Whenever the indexing is
> triggered it first makes a DB entry and then creates a catalog with
> timestamp in solr.
> The Solr cloud has 10 Nodes distributed geographically among 10
> datacenters.
> The replication factor is 2. The Solr version is 5.3.2.
> Coming to my problem - I had to write a utility to ensure that the DB
> insert timestamp matches closely to the Solr index timestamp wherein I can
> ensure that if the difference between DB timestamp and Solr Index tinestamp
> is <= 2 hrs , we have fresh index. The new index contains revised prices of
> products or offers etc which are critical to be updated as in when they
> come. Hence this utility is to track that the required updates have been
> successfully made.
> I used *CLUSTERSTATUS* api for this task. It is serving the purpose well so
> far , but pretty recently our solr cloud started complaining of strange
> things because of which the *CLUSTERSTATUS* api keeps returning as error.
>
> The error claims to be of missing config & sometime missing collections
> like.
>
> org.apache.solr.common.SolrException: Could not find collection :
> > 1785-1520548816454
>
> org.apache.solr.common.SolrException: Could not find collection :
> 1785-1520548816454
> at
> org.apache.solr.common.cloud.ClusterState.getCollection(
> ClusterState.java:165)
> at
> org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(
> ClusterStatus.java:110)
> at
> org.apache.solr.handler.admin.CollectionsHandler$
> CollectionOperation$19.call(CollectionsHandler.java:614)
> at
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> CollectionsHandler.java:166)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:143)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> HttpSolrCall.java:678)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:215)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:179)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>
> The other times it would complain of missing the config for same or
> different client id- timestamp like :
>
> 1532-1518669619526_shard1_replica3:
> org.apache.solr.common.cloud.ZooKeeperException:org.apache.
> solr.common.cloud.ZooKeeperException:
> Specified config does not exist in ZooKeeper:1532-1518669619526I
>
> I would really appreciate if :
>
>
>1. Someone can possibly guide me as to whats going on Solr Cloud
>2. If CLUSTERSTATUS is the right pick to build such utility. Do we
> have any other option?
>
>
> Thanks for any pointers and suggestions.
>
> Appreciate your attention looking this through.
>
> Atita
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

2018-03-12 Thread Hendrik Haddorp

Hi,

are your collections using stateFormat 1 or 2? In version 1 all state
was stored in one file while in version 2 each collection has its own
state.json. I assume that in the old version it could happen that the
common file still contains state for a collection that was deleted. So I
would read out the clusterstate.json in ZooKeeper and manually verify
its content.

regards,
Hendrik

On 11.03.2018 19:27, Atita Arora wrote:

Hi ,

I am working on an application which involves working on a highly
distributed Solr cloud environment. The application supports multi-tenancy
and we have around 250-300 collections on Solr where each client has their
own collection with a new shard being created as clientid-
where the timestamp is whenever the new data comes in for the client
(typically every 4-8 hrs) , the reason for this convention is to make sure
when the Indexes are being built (on demand) the timestamp matches closely
to the time when the last indexing was run (the earlier shard is
de-provisioned as soon as the new one is created). Whenever the indexing is
triggered it first makes a DB entry and then creates a catalog with
timestamp in solr.
The Solr cloud has 10 Nodes distributed geographically among 10 datacenters.
The replication factor is 2. The Solr version is 5.3.2.
Coming to my problem - I had to write a utility to ensure that the DB
insert timestamp matches closely to the Solr index timestamp wherein I can
ensure that if the difference between DB timestamp and Solr Index tinestamp
is <= 2 hrs , we have fresh index. The new index contains revised prices of
products or offers etc which are critical to be updated as in when they
come. Hence this utility is to track that the required updates have been
successfully made.
I used *CLUSTERSTATUS* api for this task. It is serving the purpose well so
far , but pretty recently our solr cloud started complaining of strange
things because of which the *CLUSTERSTATUS* api keeps returning as error.

The error claims to be of missing config & sometime missing collections
like.

org.apache.solr.common.SolrException: Could not find collection :

1785-1520548816454

org.apache.solr.common.SolrException: Could not find collection :
1785-1520548816454
at
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:165)
at
org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(ClusterStatus.java:110)
at
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$19.call(CollectionsHandler.java:614)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:166)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:678)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)

The other times it would complain of missing the config for same or
different client id- timestamp like :

1532-1518669619526_shard1_replica3:
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
Specified config does not exist in ZooKeeper:1532-1518669619526I

I would really appreciate if :

1. Someone can possibly guide me as to whats going on Solr Cloud
2. If CLUSTERSTATUS is the right pick to build such utility. Do we
have any other option?

Thanks for any pointers and suggestions.

Appreciate your attention looking this through.

Atita

CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

2018-03-11 Thread Atita Arora

Hi ,

The error claims to be of missing config & sometime missing collections
like.

org.apache.solr.common.SolrException: Could not find collection :
> 1785-1520548816454

The other times it would complain of missing the config for same or
different client id- timestamp like :

1532-1518669619526_shard1_replica3:
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
Specified config does not exist in ZooKeeper:1532-1518669619526I

I would really appreciate if :

1. Someone can possibly guide me as to whats going on Solr Cloud
2. If CLUSTERSTATUS is the right pick to build such utility. Do we
have any other option?

Thanks for any pointers and suggestions.

Appreciate your attention looking this through.

Atita

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

Re: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

4 matches

Site Navigation

Mail list logo

Footer information