Re: Migrate from sol 5.3.1 to 7.5.0
Basically, you have to re-index whenever you change the schema, with very few exceptions. Some changes cause exceptions. Some changes just fail to return the correct results. Etc. You can _add_ completely new fields without reindexing, but they won't have any values for existing documents. You can _remove_ fields completely, but they'll still be present in the old documents. "When in doubt, re-index" On Thu, Feb 14, 2019 at 5:39 PM ramyogi wrote: > > Do we need to reindex if we change synonymQueryStyle values for a fieldType ? > I hope not. > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
Do we need to reindex if we change synonymQueryStyle values for a fieldType ? I hope not. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
Hi Eric, Found the reason and all shard request to go /select flow solves the problem. Regarding In SOLR 7 when relevancy added for the search it is not working (not expected results) for the above fieldtype but same works fine for /select because it uses Lucene Parser but our flow uses EDISMAX. But 5.3.1 works fine both cases. As soon as we change as below, synonymQueryStyle="as_same_term" Both cases works in 7.5.0 flow. Is there any intentional changes for the above scenario in SOLR 7.5.0 where I can refer ? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
I can’t find the original post right now, but putting a load balancer in front of Zookeeper is a really bad idea. Do not do that. There is a stateful protocol between one client and one Zookeeper node. This is not a stateless protocol that you can just bounce around between servers. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 14, 2019, at 7:42 AM, Erick Erickson wrote: > > bq. 2. Did you take the solrconfig that came with 7.5 and modify it rather > than copy your 5.3 solrconfig? > No, We have adjusted existing 5.3.1 config when we used for couple years to > SOLR 7.5.0. > > I don't think this is the root of your problem, but this is always > suspect. Not only > can the format of solrconfig change, but certain values don't make any sense, > e.g. LuceneMatchVersion would be 2 major versions back, which is unsupported. > You may well have changed _that_ value, but can you guarantee all > _other_ changes > are accounted for? > > So I'd takethe 7.x solrconfig (and schema for that matter) and overlay > your changes > rather than try to update the 5x configs. > > Best, > Erick > > On Thu, Feb 14, 2019 at 1:59 AM Jan Høydahl wrote: >> >> You may of course ignore the warning in the UI, it is just a warning >> intended to help you avoid mis-configurations. >> But there may be side effects of placing a load balancer in between client >> and zk cluster, see >> https://stackoverflow.com/questions/30905007/load-balancer-with-zookeeper >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >>> 14. feb. 2019 kl. 01:24 skrev ramyogi : >>> >>> Thanks Jan, I am unaware how our Devops team decided this But this was >>> working well without any issues with SOLR 5.3.1 for couple of years. Just >>> wanted to make any changes in SOLR7 mandates . >>> >>> >>> >>> -- >>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >>
Re: Migrate from sol 5.3.1 to 7.5.0
bq. 2. Did you take the solrconfig that came with 7.5 and modify it rather than copy your 5.3 solrconfig? No, We have adjusted existing 5.3.1 config when we used for couple years to SOLR 7.5.0. I don't think this is the root of your problem, but this is always suspect. Not only can the format of solrconfig change, but certain values don't make any sense, e.g. LuceneMatchVersion would be 2 major versions back, which is unsupported. You may well have changed _that_ value, but can you guarantee all _other_ changes are accounted for? So I'd takethe 7.x solrconfig (and schema for that matter) and overlay your changes rather than try to update the 5x configs. Best, Erick On Thu, Feb 14, 2019 at 1:59 AM Jan Høydahl wrote: > > You may of course ignore the warning in the UI, it is just a warning intended > to help you avoid mis-configurations. > But there may be side effects of placing a load balancer in between client > and zk cluster, see > https://stackoverflow.com/questions/30905007/load-balancer-with-zookeeper > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 14. feb. 2019 kl. 01:24 skrev ramyogi : > > > > Thanks Jan, I am unaware how our Devops team decided this But this was > > working well without any issues with SOLR 5.3.1 for couple of years. Just > > wanted to make any changes in SOLR7 mandates . > > > > > > > > -- > > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: Migrate from sol 5.3.1 to 7.5.0
You may of course ignore the warning in the UI, it is just a warning intended to help you avoid mis-configurations. But there may be side effects of placing a load balancer in between client and zk cluster, see https://stackoverflow.com/questions/30905007/load-balancer-with-zookeeper -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 14. feb. 2019 kl. 01:24 skrev ramyogi : > > Thanks Jan, I am unaware how our Devops team decided this But this was > working well without any issues with SOLR 5.3.1 for couple of years. Just > wanted to make any changes in SOLR7 mandates . > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
Thanks Erick, I was waiting your Day time appears to get suggestion because
you are very spontaneous for this great open source community.
1. Did you recompile and redistriburte your custom component?
* Yes*
2. Did you take the solrconfig that came with 7.5 and modify it rather
than copy your 5.3 solrconfig?
No, We have adjusted existing 5.3.1 config when we used for couple years to
SOLR 7.5.0.
3. Did you reindex from scratch with 7x? Lucene/Solr guarantees only
one major version back-compat.
Yes, We have done Full Reindexing. No issues noticed with reindex.
All works as expected in the flow of /select request handler. Noticed one
stuff below changed in SOLR 7.5.0
All Shard Requests are distributed back to custom component flow instead
default components.
// for distributed queries that don’t include shards.qt, use the original
path
// as the default but operators need to update their
luceneMatchVersion to enable
// this behavior since it did not work this way prior to 5.1
if
(req.getCore().getSolrConfig().luceneMatchVersion.onOrAfter(Version.LUCENE_5_1_0))
{
String reqPath = (String) req.getContext().get(PATH);
if (!“/select”.equals(reqPath)) {
params.set(CommonParams.QT, reqPath);
} // else if path is /select, then the qt gets passed thru
if set
} else {
// this is the pre-5.1 behavior, which translates to
sending the shard request to /select
params.remove(CommonParams.QT);
}
Below some portion of debug log.
Enabled Debug and noticed that for the custom component handler flow for all
shard request,
"PARSE_QUERY":{
"http://<>_shard26_replica_n9/":{
"QTime":"27",
"ElapsedTime":"28",
"RequestPurpose":"GET_TERM_STATS",
"NumFound":"<>",
"Response":"{responseHeader={zkConnected=true,xqs=best,info={},status=0,QTime=27},response={numFound=
"EXECUTE_QUERY":{
"http://<>_shard3_replica_n27/":{
"QTime":"26",
"ElapsedTime":"26",
"RequestPurpose":"GET_TOP_IDS,GET_FIELDS,GET_DEBUG,SET_TERM_STATS",
"NumFound":"<>",
"Response":"{responseHeader={zkConnected=true,xqs=best,info={},status=0,QTime=26},response={numFound=
*But when Default(/select) flow goes,*
"PARSE_QUERY":{
"http://<>_shard24_replica_n6/":{
"QTime":"0",
"ElapsedTime":"1",
"RequestPurpose":"GET_TERM_STATS",
"Response":"{responseHeader={zkConnected=true,status=0,QTime=0,params={df=all,distrib=false,debug=[false,
timing, track],shards.purpose=3276
}
"EXECUTE_QUERY":{
"http://<>_shard30_replica_n1/":{
"QTime":"17",
"ElapsedTime":"19",
"RequestPurpose":"GET_TOP_IDS,SET_TERM_STATS",
"NumFound":"316605",
"Response":"{responseHeader={zkConnected=true,status=0,QTime=17,params={df=all,distrib=false,fl=[id,
score],shards.purpose=16388
is there any documentation about to read all these flow purpose and
understand better
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
Thanks Jan, I am unaware how our Devops team decided this But this was working well without any issues with SOLR 5.3.1 for couple of years. Just wanted to make any changes in SOLR7 mandates . -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
1. Did you recompile and redistriburte your custom component? 2. Did you take the solrconfig that came with 7.5 and modify it rather than copy your 5.3 solrconfig? 3. Did you reindex from scratch with 7x? Lucene/Solr guarantees only one major version back-compat. Best, Erick On Wed, Feb 13, 2019 at 7:26 AM Jan Høydahl wrote: > > You need to list all zk hosts in zkHost, that’s part of the design for ZK, > clients will open connections to all nodes and be able to fail over if > trouble. > > Where did you get the CNAME advice? > > Jan Høydahl > > > 13. feb. 2019 kl. 05:56 skrev ramyogi : > > > > We are migrating SOLR version, We used 3 ZK hosts that configured to SOLR as > > ZK connection string: zookeeper.solrtest.net:2181/test-config > > Ensemble size: 1 > > Ensemble mode: ensemble > > zookeeper.solrtest.net:2181 > > oktrue > > clientPort2181 > > zk_server_statefollower > > zk_version3.4.5 > > zk_approximate_data_size9902464 > > zk_znode_count1734 > > zk_num_alive_connections43 > > serverId4 > > electionPort3888 > > quorumPort2888 > > > > zookeeper.solrtest.net . is nothing but CNAME of all three ZKHost together. > > > > In solr admin console we see > > Errors: > > We do not have a leader > > > > > > And we are using custom component before query goes to "query" component in > > proper order execution but pagination is not working after some limit for > > example :start:599 results comes but start:600 docs[] empty in the response. > > Results counts are reduced for some queries fields(combined) if we use our > > custom component order but works if we use /select. > > > > > > > > -- > > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Migrate from sol 5.3.1 to 7.5.0
You need to list all zk hosts in zkHost, that’s part of the design for ZK, clients will open connections to all nodes and be able to fail over if trouble. Where did you get the CNAME advice? Jan Høydahl > 13. feb. 2019 kl. 05:56 skrev ramyogi : > > We are migrating SOLR version, We used 3 ZK hosts that configured to SOLR as > ZK connection string: zookeeper.solrtest.net:2181/test-config > Ensemble size: 1 > Ensemble mode: ensemble > zookeeper.solrtest.net:2181 > oktrue > clientPort2181 > zk_server_statefollower > zk_version3.4.5 > zk_approximate_data_size9902464 > zk_znode_count1734 > zk_num_alive_connections43 > serverId4 > electionPort3888 > quorumPort2888 > > zookeeper.solrtest.net . is nothing but CNAME of all three ZKHost together. > > In solr admin console we see > Errors: > We do not have a leader > > > And we are using custom component before query goes to "query" component in > proper order execution but pagination is not working after some limit for > example :start:599 results comes but start:600 docs[] empty in the response. > Results counts are reduced for some queries fields(combined) if we use our > custom component order but works if we use /select. > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Migrate from sol 5.3.1 to 7.5.0
We are migrating SOLR version, We used 3 ZK hosts that configured to SOLR as ZK connection string: zookeeper.solrtest.net:2181/test-config Ensemble size: 1 Ensemble mode: ensemble zookeeper.solrtest.net:2181 ok true clientPort 2181 zk_server_state follower zk_version 3.4.5 zk_approximate_data_size9902464 zk_znode_count 1734 zk_num_alive_connections43 serverId4 electionPort3888 quorumPort 2888 zookeeper.solrtest.net . is nothing but CNAME of all three ZKHost together. In solr admin console we see Errors: We do not have a leader And we are using custom component before query goes to "query" component in proper order execution but pagination is not working after some limit for example :start:599 results comes but start:600 docs[] empty in the response. Results counts are reduced for some queries fields(combined) if we use our custom component order but works if we use /select. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
