Re: Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
Thanks for the replies. Wanted to make sure that I am not missing on something already available. Now that everything is clear, I may have to go for application re-design or choose an approach, not so elegant, as suggested. Thanks all. -Gajanan -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
An unelegant alternative is to back up and delete a collection in lieu of unload, restore it in lieu of load... Ilan On Sun, Nov 15, 2020 at 6:56 PM Erick Erickson wrote: > I don’t really have any good alternatives. There’s an open JIRA for > this, see: SOLR-6399 > > This would be a pretty big chunk of work, which is one of the reasons > this JIRA has languished… > > Sorry I can’t be more helpful, > Erick > > > On Nov 15, 2020, at 11:00 AM, Gajanan wrote: > > > > Hi Erick, thanks for the reply. > > I am working on a application where a solr collection is being created > per > > usage of application accumulating lot of them over period of time . In > order > > to keep memory requirements under control, I am unloading collections > not in > > current usage and loading them whenever required. > > This was working in non cloud mode with coreAdmin APIs. Now because of > > scaling requirements we want to shift to SolrCloud mode. we want to > continue > > with same application design. can you suggest, how to implement a similar > > solution in SolrCloud context. > > > > -Gajanan > > > > > > > > -- > > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html > >
Re: Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
I don’t really have any good alternatives. There’s an open JIRA for this, see: SOLR-6399 This would be a pretty big chunk of work, which is one of the reasons this JIRA has languished… Sorry I can’t be more helpful, Erick > On Nov 15, 2020, at 11:00 AM, Gajanan wrote: > > Hi Erick, thanks for the reply. > I am working on a application where a solr collection is being created per > usage of application accumulating lot of them over period of time . In order > to keep memory requirements under control, I am unloading collections not in > current usage and loading them whenever required. > This was working in non cloud mode with coreAdmin APIs. Now because of > scaling requirements we want to shift to SolrCloud mode. we want to continue > with same application design. can you suggest, how to implement a similar > solution in SolrCloud context. > > -Gajanan > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
Hi Erick, thanks for the reply. I am working on a application where a solr collection is being created per usage of application accumulating lot of them over period of time . In order to keep memory requirements under control, I am unloading collections not in current usage and loading them whenever required. This was working in non cloud mode with coreAdmin APIs. Now because of scaling requirements we want to shift to SolrCloud mode. we want to continue with same application design. can you suggest, how to implement a similar solution in SolrCloud context. -Gajanan -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
As stated in the docs, using the core admin API when using SolrCloud is not recommended, for just reasons like this. While SolrCloud _does_ use the Core Admin API, it’s usage has to be very precise. You apparently didn’t heed this warning in the UNLOAD command for the collections API: "Unloading all cores in a SolrCloud collection causes the removal of that collection’s metadata from ZooKeeper.” This latter is what the “non legacy mode…” message is about. In earlier versions of Solr, the ZK information was recreated when Solr found a core.properties file, but that had its own problems so was removed. Your best bet now is to wipe your directories, create a new collection and re-index. If you absolutely can’t reindex: 0> save away one index directory from every shard, it doesn’t matter which. 1> create the collection, with the exact same number of shards and a replicationFactor of 1 2> shut down all the Solr instances 3> copy the index directory from <0> to ’the right place”. For instance, if you have a collection blah, you’ll have some directory like blah_shard1_replica_n1/data/index. It’s critical that you replace the contents of data/index with the contents of the directory saved in <0> from the _same_ shard, shard1 in this example. 4> start your Solr instances back up 5> use ADDREPLICA to build out the collection to have as many replicas as you need. Good luck! Erick > On Nov 12, 2020, at 6:32 AM, Gajanan wrote: > > I have unloaded all cores of a collection in SolrCloud (8.x.x ) using > coreAdmin APIs as UNLOAD collection is not available in collections API. Now > I want reload the unloaded collection using APIs only. > When trying with coreAdmin APIs I am getting "Non legacy mode CoreNodeName > not found." > When trying with collections APIs it is reloaded but shows no cores > available. > > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Unloading and loading a Collection in SolrCloud with external Zookeeper ensemble
I have unloaded all cores of a collection in SolrCloud (8.x.x ) using coreAdmin APIs as UNLOAD collection is not available in collections API. Now I want reload the unloaded collection using APIs only. When trying with coreAdmin APIs I am getting "Non legacy mode CoreNodeName not found." When trying with collections APIs it is reloaded but shows no cores available. -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html