RE: Autoscaling working Examples
Hi Florian, We have a Solr Cloud installation with 2 nodes, each in a separate Availability Zone. We run one shard for each collection, with replicationFactor set to 2 and autoAddReplica set to True. These are the only settings we make other than using our config set. Typical settings for our collections: Shard Count:1 configName ourConfig replicationFactor 2 maxShardsperNode1 router: compositeId autoAddReplicas:True Hope this helps, Shane -Original Message- From: Florian Gleixner Sent: Thursday, February 18, 2021 9:03 AM To: solr-user@lucene.apache.org Subject: Autoscaling working Examples WARNING: This email originated from outside Municode. DO NOT CLICK on any attachments or links from unknown senders or unexpected emails. Always check the sender's display name and email address are correct before you communicate. Hi, i'm trying ot get Autoscaling running, but i always get more and more repicas. Some shards have two replicas, some have up to 7 Replicas while the replicationFactor of the collection is 2. Is there a working example, that keeps shard replicas distributed over 2 availability zones with exactly one replica in every zone? Florian
Replication not occurring to newly added SOLRCloud nodes
Main info: SOLRCloud 7.7.3, Zookeeper 3.4.14
I have a 2 node SOLRCloud installation, 3 zookeeper instances, configured in
AWS to autoscale. I am currently testing with 9 collections. My issue is that
when I scale out and a node is added to the SOLRCloud cluster,
I get replication to the new node from only one of the 9 collections.
https://i.imgur.com/xR7PQxf.gif
autoAddReplicas=true for each of the 9 collections.
I have set the following rules:
# maintain 2 replicas of each shard
curl -X POST "http://localhost:8983/solr/admin/autoscaling"; --data-binary \
'{
"set-cluster-policy": [
{ "replica": "2","shard": "#EACH", "node": "#ANY" }
]
}'
# listen for a nodeAdded event and upon 5 seconds of receiving begin
replicating our existing collection(s) to the new node.
curl -X POST "http://localhost:8983/solr/admin/autoscaling"; --data-binary \
'{
"set-trigger": {
"name": "node_added_trigger",
"event": "nodeAdded",
"waitFor": "5s",
"preferredOperation": "ADDREPLICA"
}
}'
All of the collections were set up identically. What else do I need to do to
automatically force replication to nodes when they are added to the cluster?
Thanks in advance,
Shane Brooks
RE: Can create collections with Drupal 8 configset
> The class that doesn't load (in your error message) is not located in the
> icu4j jar. It is located in the lucene-analyzers-icu-X.Y.Z.jar file, which
> is found in the contrib/analysis-extras/lucene-libs
subdirectory.
Thanks Shawn, this was exactly my issue. That subdirectory was missing
completely. I appreciate all your help!
Shane
-Original Message-
From: Shawn Heisey
Sent: Sunday, August 9, 2020 3:24 PM
To: solr-user@lucene.apache.org
Subject: Re: Can create collections with Drupal 8 configset
On 8/9/2020 8:11 AM, Shane Brooks wrote:
> Thanks Shawn. The way we have it configured presently is as follows:
> icu4j.jar is located in
> /opt/solr/contrib/analysis-extras/lib/icu4j-62.1.jar
>
> solrconfig.xml contains:
>
> regex=".*\.jar" />
>
> Which should load the jar at startup, correct?
I do not know if that path spec is right or not. It might be.
The class that doesn't load (in your error message) is not located in the icu4j
jar. It is located in the lucene-analyzers-icu-X.Y.Z.jar file, which is found
in the contrib/analysis-extras/lucene-libs
subdirectory. That jar also needs the icu4j jar.
If the same class is loaded more than once, it probably won't work. I know for
sure from experience that this is the case for the Lucene ICU classes. That's
the biggest reason I use the ${solr.home}/lib directory
-- so I am sure that each extra jar is only loaded once. That directory does
not exist until you create it.
Thanks,
Shawn
RE: Can create collections with Drupal 8 configset
Thanks Shawn. The way we have it configured presently is as follows: icu4j.jar is located in /opt/solr/contrib/analysis-extras/lib/icu4j-62.1.jar solrconfig.xml contains: Which should load the jar at startup, correct? Is there anything significant about the mime-type error? Expected mime type application/octet-stream but got text/html Thanks, Shane -Original Message- From: Shawn Heisey Sent: Sunday, August 9, 2020 3:00 AM To: solr-user@lucene.apache.org Subject: Re: Can create collections with Drupal 8 configset On 8/8/2020 10:31 PM, Shane Brooks wrote: > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:E > rror from server at http://192.168.xx.xx:8983/solr: Expected mime type > application/octet-stream but got text/html. \n\n http-equiv=\"Content-Type\" > content=\"text/html;charset=utf-8\"/>\nError 500 Server > Error\n\nHTTP ERROR 500\nProblem > accessing /solr/admin/cores. Reason:\n Server > ErrorCaused by:java.lang.NoClassDefFoundError: > org/apache/lucene/collation/ICUCollationKeyAnalyzer\n\tat > > > I haven't found anything on Google except that ICUCollationKeyAnalyzer > depends on the icu4j library, which I verified is part of the SOLR > package. The icu4j library and the related Lucene jars for ICU capability are not part of Solr by default. They can be found in the "contrib" in the Solr download ... but you must add the jars to Solr if you intend to use any contrib capability. The best way I have found to add custom jars to Solr is to create a "lib" directory under the location designated as the solr home and place the jars there. All jars found in that directory will be automatically loaded and will be available to all cores. An alternate way to load jars is with the directive in solrconfig.xml ... but I don't recommend that approach. Thanks, Shawn
Can create collections with Drupal 8 configset
Hello, Main info: Drupal 8, SOLRCloud 7.7.3, Zookeeper 3.4.14, search_api 8.x-1.17, search_api_solr 4.14 Migrating from a standalone SOLR 7.7.3 instance where everything works as expected. Drupal 7 configset gives us no issue on SOLRCloud. Issue: When I try to create a new collection using the Drupal 8 configset via POST: http://solr.ourcompany.com:8983/solr/admin/collections?action=CREATE&name=ourcollection&numShards=1&replicationFactor=2&autoAddReplicas=true&collection.configName=d8configset&rule=replica:<2,node:* I get the following error: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://192.168.xx.xx:8983/solr: Expected mime type application/octet-stream but got text/html. \n\n\nError 500 Server Error\n\nHTTP ERROR 500\nProblem accessing /solr/admin/cores. Reason:\n Server ErrorCaused by:java.lang.NoClassDefFoundError: org/apache/lucene/collation/ICUCollationKeyAnalyzer\n\tat I haven't found anything on Google except that ICUCollationKeyAnalyzer depends on the icu4j library, which I verified is part of the SOLR package. Can anyone help me with this issue? Is there anything else I can provide? Thanks, Shane Shane Brooks Systems Administrator Phone 800.262.2633 x 7465 Email sh...@municode.com<mailto:sh...@municode.com> [A close up of a sign Description automatically generated] Connecting you and your community through the circle of governance: < Meeting Management<https://www.municode.com/meetings/> < Website Design<https://www.municodeweb.com/> < Policy & Procedure Software < Legislative Publishing Software<https://www.municode.com/self-publishing/> < Document Archival Tool<https://www.municode.com/online-hosting/#munidocs> < Codification Services<https://www.municode.com/codification-and-recodification/>
