Re: Can we have multiple Spellcheck Components under /select handler

[Rajesh Hazari]

Hi,

You can have more than one spellcheck components after you have the
spellcheck components in select request handler,
you have still which spellcheck you want to use you could mention in your
query params
for ex:
http://
:8983/solr//select/?q=text=default=wordbreak=true

or define as defualts in you select handler:

   
 
true
default
wordbreak
.

elevator
spellcheck



*Rajesh.*


On Wed, Dec 30, 2020 at 7:51 AM rashi gandhi 
wrote:

> Hi All
>
> I am trying to configure multiple spellcheck components. I defined two
> searchComponents in my solrconfig.xml, let's say  and
> .
> And added above both components under /select request handler with default
> required attributes.
>
> >  elevator spellcheck <
> > str>spellcheck1 
>
>
> However, I can not see spellcheck1 coming in response. (when I set
> /select?spellcheck1=true=).
>
> Can't we configure multiple spellcheck components with different names in
> Solr.
>

Re: Problem with spellchecker component

[Rajesh Hazari]

What spellcheckers you have in your collection configs,
do you have any of these

 
wordbreak
solr.WordBreakSolrSpellChecker
.
.
.
.
 

  
default
textSpell
solr.IndexBasedSpellChecker
.
.
.
.
.
 

we have come up with these spellcheckers which works with our schema
definitions.

*Rajesh**.*

On Fri, Oct 7, 2016 at 2:36 PM, la...@2locos.com  wrote:

> I'm using Spellcheck component and it doesn't show me any error for
> combination of words with error, I want to know if it just work on one word
> or it also works on combination of words?and if so what should I do to
> makes it work?
>
> Ladan Nekuii
> Web Developer
> 2locos
> 300 Frank H. Ogawa Plaza, Suite 234
> Oakland, CA 94612
> Tel: 510-465-0101
> Fax: 510-465-0104
> www.2locos.com
>
>

Re: solr 5 leaving tomcat, will I be the only one fearing about this?

[Rajesh Hazari]

Renee,
I dont understand what would be the difficulties of putting your 3rd party
distributions in some contrib folders and
import them in your solr configs and your processors should get loaded
using solr class loader.

we have used our custom based synonym processor putting the jar in contrib
folder and imported in our solrconfig.xml

for ex: 
placed our custom jar file in
${solr_home}/contrib/analysis-extras/lucene-libs/ folder.
with solr 4.9 version using jetty.
Before we upgraded from tomcat based solr deployment we used to have our
custom jar file in solr.war/WEB-INF/lib.

I'm not sure if this answers your question, this to give you comfortability
of solr with jetty, which is more preferred deployment.


*Rajesh**.*

On Fri, Oct 7, 2016 at 3:09 PM, John Bickerstaff 
wrote:

> I won't speak for the committers, but I'm guessing you won't find a lot of
> support for the idea of continuing to provide a WAR file with the standard
> SOLR releases...
>
> I feel for you and your situation however - I've had to wrestle with a
> number of situations where a somewhat monolithic architecture was disturbed
> by newer ways of doing things...
>
> That leaves 2 options...
>
> A self-maintained build of the latest SOLR into a WAR file (challenging in
> it's own way) or the very least amount of code necessary to allow your
> Tomcat-based tools to talk to a 6.x Solr server running in Jetty...
>
> I imagine something better than re-writing all your code can be done,
> although I also don't think you can get away with no new code either...  At
> a high level, some kind of API/message bus/interface comes to mind, but I
> don't know enough about your situation to be able to guess what might be a
> good approach.
>
> If you're interested in a discussion about how to approach the change, I'd
> be happy to offer ideas, but I'd need to know how your other tools
> currently talk to Solr...  Of course, you may not want to even have that
> discussion if the task is just to big...
>
> On Fri, Oct 7, 2016 at 9:33 AM, Renee Sun  wrote:
>
> > Thanks ... but that is an extremely simplified situation.
> >
> > We are not just looking for Solr as a new tool to start using it.
> >
> > In our production, we have cloud based big data indexing using Solr for
> > many
> > years. We have developed lots business related logic/component deployed
> as
> > webapps working seamlessly with solr.
> >
> > I will give you a simple example, we purchased multi-lingual processors
> > (and
> > many other 3rd parties) which we integrated with solr by carefully deploy
> > the libraries (e.g.) in the tomcat container so they work together. This
> > basically means we have to rewrite all those components to make it work
> > with
> > solr 5 or 6.
> >
> > In my opinion, for those solr users like our company, it will really be
> > beneficial if Solr could keep supporting deploying a war and maintain
> > parallel support with its new standalone release, although this might be
> > too
> > much work?
> >
> > Thanks
> > Renee
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.
> > nabble.com/solr-5-leaving-tomcat-will-I-be-the-only-one-
> > fearing-about-this-tp4300065p4300202.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>

requestlog jetty param in solr 5.x

[Rajesh Hazari]

Hi All,

solr version: 5.5.0

I was checking to see if there is any quick solution for embedded jetty can
log request access logs too.

After some googling i found documentation (here
)
that there is configuration option param --add-to-startd=requestlog which
it looks for and creates and access logs in {$jetty.base}

Initially i started using -a option in startup

for ex:
./solr restart -m 4g -s ${solr.home} -V -a "--add-to-startd=requestlog"  ,
which did not work
basically -a option is used only to pass solr params

Then i went on to change my ./solr script with
 SOLR_JETTY_CONFIG+=("--add-to-startd=requestlog")
.
 and added echo statement which prints the jetty config params(with
verbose option -V)

if [ "$SOLR_JETTY_CONFIG" != "" ]; then
  echo -e "SOLR_JETTY_CONFIG= ${SOLR_JETTY_CONFIG[@]}"
fi

when i restart the solr server
./solr restart -m 4g -s ${solr.home} -V
the above change prints the below jetty config params, but server does not
start.

can't we pass extra arguments to jetty server from solr start up script or
is there other place that we have to change?


*Thanks,*
*Rajesh**.*

Re: Questions on SolrCloud core state, when will Solr recover a "DOWN" core to "ACTIVE" core.

[Rajesh Hazari]

Hi Li,

Do you see timeouts liek "CLUSTERSTATUS the collection time out:180s"
if its the case, this may be related to
https://issues.apache.org/jira/browse/SOLR-7940,
and i would say either use the patch file or upgrade.


*Thanks,*
*Rajesh,*
*8328789519,*
*If I don't answer your call please leave a voicemail with your contact
info, *
*will return your call ASAP.*

On Thu, Apr 21, 2016 at 6:02 AM, YouPeng Yang 
wrote:

> Hi
>We have used Solr4.6 for 2 years,If you post more logs ,maybe we can
> fixed it.
>
> 2016-04-21 6:50 GMT+08:00 Li Ding :
>
> > Hi All,
> >
> > We are using SolrCloud 4.6.1.  We have observed following behaviors
> > recently.  A Solr node in a Solrcloud cluster is up but some of the cores
> > on the nodes are marked as down in Zookeeper.  If the cores are parts of
> a
> > multi-sharded collection with one replica,  the queries to that
> collection
> > will fail.  However, when this happened, if we issue queries to the core
> > directly, it returns 200 and correct info.  But once Solr got into the
> > state, the core will be marked down forever unless we do a restart on
> Solr.
> >
> > Has anyone seen this behavior before?  Is there any to get out of the
> state
> > on its own?
> >
> > Thanks,
> >
> > Li
> >
>

Re: Load Resource from within Solr Plugin

[Rajesh Hazari]

Max,
Have you looked in External file field which is reload on every hard commit,
only disadvantage of this is the file (personal-words.txt) has to be placed
in all data folders in each solr core,
for which we have a bash script to do this job.

https://cwiki.apache.org/confluence/display/solr/Working+with+External+Files+and+Processes

Ignore this if this does not meets your requirement.

*Rajesh**.*

On Wed, Mar 30, 2016 at 1:21 PM, Chris Hostetter 
wrote:

> :
> :  : regex=".*\.jar" />
>
> 1) as a general rule, if you have a  delcaration which includes
> "WEB-INF" you are probably doing something wrong.
>
> Maybe not in this case -- maybe "search-webapp/target" is a completley
> distinct java application and you are just re-using it's jars.  But 9
> times out of 10, when people have a  WEB-INF path they are trying to load
> jars from, it's because they *first* added their jars to Solr's WEB_INF
> directory, and then when that didn't work they added the path to the
> WEB-INF dir as a  ... but now you've got those classes being loaded
> twice, and you've multiplied all of your problems.
>
> 2) let's ignore the fact that your path has WEB-INF in it, and just
> assume it's some path to somewhere where on disk that has nothing to
> do with solr, and you want to load those jars.
>
> great -- solr will do that for you, and all of those classes will be
> available to plugins.
>
> Now if you wnat to explicitly do something classloader related, you do
> *not* want to be using Thread.currentThread().getContextClassLoader() ...
> because the threads that execute everything in Solr are a pool of worker
> threads that is created before solr ever has a chance to parse your  /> directive.
>
> You want to ensure anything you do related to a Classloader uses the
> ClassLoader Solr sets up for plugins -- that's available from the
> SolrResourceLoader.
>
> You can always get the SolrResourceLoader via
> SolrCore.getSolrResourceLoader().  from there you can getClassLoader() if
> you really need some hairy custom stuff -- or if you are just trying to
> load a simple resource file as an InputStream, use openResource(String
> name) ... that will start by checking for it in the conf dir, and will
> fallback to your jar -- so you can have a default resource file shipped
> with your plugin, but allow users to override it in their collection
> configs.
>
>
> -Hoss
> http://www.lucidworks.com/
>

Re: How to boost query based on result of subquery?

[Rajesh Hazari]

Hi Ed,

Did you look into ExternalFilefield type (for ex: with name ::
 position_external_field  in your schema), which can be used to map to your
field (for ex position, hope these are not changed very often) and then use
position_external_field in your boost function.

This can be used if you can comeup with unique field values for position
field as this is application specific field,
this can be changed to something like, if these are finite.
position_5=5
position_25=25
position_55=55

for ex: =custom_function(field(query_position_external),
field(position_external))

for more info refer wiki

.

pros:
the value of this field can be refreshed with every newsearcher and
firstsearcher
using



Cons: This file has to reside in data folder of each replica,
   updating of this file will have to be some bash script.

*Please ignore if this may not work for you.*

*Rajesh**.*

On Fri, Feb 19, 2016 at 1:19 PM, Edward P  wrote:

> Hello,
>
> I am using Solr 5.4.0, one collection, multiple shards with replication.
> Sample documents:
> {
> "item_id": "30d1e667",
> "date": "2014-01-01",
> "position": "5",
> "description": "automobile license plate holder"
> }
>
> {
> "item_id": "3cf18028",
> "date": "2013-01-01",
> "position": "23",
> "description": "dinner plate"
> }
>
> {
> "item_id": "be1b2643",
> "date": "2013-06-01",
> "position": "21",
> "description": "ceramic plate"
> }
>
>
> The client sends 2 queries like this:
> (1) /select?q=item_id:30d1e667=position
> (2) /select?q=plate_position=5=custom_function($query_position,
> $position)=item_id,date,description
>
> The idea is, we have an application-specific data field "position" which we
> use to compare 2 items. The client looks up a particular item by item_id,
> gets the position data, then sends it back in the 2nd query to influence
> the ranking of items when performing a text search for "plate". Our
> custom_function is app-specific and may for example derive the boost from
> the difference of query_position and document's position.
>
> My need is: I want to combine these into one query, so the client will only
> have to send something like:
>
> /select?query_item_id=30d1e667_text=plate={… use of Solr nested
> queries, boost functions etc …}=item_id,date,description
>
> I want this to be one request so that both queries are executed against the
> same searcher (because index updates may change the position values) and so
> the details of using the "position" field are abstracted from the client.
>
> I have considered the query(subquery,default) function. This is close, but
> not exactly what I need because it returns the subquery score, not document
> values.
>
> The join query parser is also close to what I need, but I can't see how to
> use it to direct the results of a subquery into the boost function of
> another.
>
> So how can I, in a single Solr request, extract a value from the result
> document of one subquery, and pass that value into a boost function for a
> 2nd query, all using the same underlying searcher? If it's not possible
> with existing nested/sub-queries, then should I explore writing a custom
> SearchComponent, QParser, or some other plugin?
>
> thanks,
> Ed
>

Re: Data Import Handler - autoSoftCommit and autoCommit

[Rajesh Hazari]

we have this for a collection which updated every 3mins with min of 500
documents and once in a day of 10k documents in start of the day


   ${solr.autoCommit.maxTime:30}
1
true
true
 
   
  ${solr.autoSoftCommit.maxTime:6000}
   

As per solr documentation, If you have solr client to index documents,
its not suggested to use commit=true and optimize=true explicitly.

we have not tested data import handle with 10 million records.

we have settled with this config after many tests and after understanding
the need and requirements.


*Rajesh**.*

On Mon, Feb 8, 2016 at 10:15 AM, Troy Edwards 
wrote:

> We are running the data import handler to retrieve about 10 million records
> during work hours every day of the week. We are using Clean = true, Commit
> = true and Optimize = true. The entire process takes about 1 hour.
>
> What would be a good setting for autoCommit and autoSoftCommit?
>
> Thanks
>

solr suggester build issues

[Rajesh Hazari]

Solr : 4.9.x , with simple solr cloud on jetty.
JDK 1.7
num of replica : 4 , one replica for each shard
num of shards : 1

Hi All,

I have been facing below issues with solr suggester introduced in 4.7.x. Do
any one have good working solution or

buildOnCommit=true property is suggested not to use with index with more
frequent softcommits as suggested in the documentation
   https://cwiki.apache.org/confluence/display/solr/Suggester
So we have disabled this (buildOnCommit=false) and started using
buildOnOptimize=true, which was not helping us to have latest document
 suggestion (with frequent softcommits),
as hardly there was one optimize each day. (we have default optimize
setting in solrconfig)
So we have disabled buildOnOptimize (buildOnOptimize=false)

As suggested in the documentation, as of now, we came up with cron jobs to
build the suggester for every hour.
These jobs are doing their job, i.e, we are having the latest suggestions
available every hour, below are issues that we have this implementation.

*Issue#1* : Suggest built url i.e,
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true*  if
issued to one replica of solr cloud does not build suggesters in all of the
replicas in solrcloud.
Resolution: For which we have separate cron jobs on each of the
solr instance having the build call to build the suggester, below is the
raw pictorial representation of this impl
(which is not the best implementation which has
many flaws)


*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true*
* |*
* |--
suggestcron.job.sh http://suggestcron.job.sh (on solr1.aws.instance)*


*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true*
* |*
* |--
suggestcron.job.sh http://suggestcron.job.sh (on solr2.aws.instance)*
*  .. similar for other solr nodes*
* We will be coming up with single script to go this for all
collection later.*

we were bit happy that we are having a updated suggester in all of the
instances, *which is not!*

*The issue#2 the suggester built on all solr nodes were not consistent as
the solr core in each solr replica have difference in max-docs and
num-docs *
*(which is quiet normal **with frequent softcommits , when updates mostly
have the same documents updated with different data, **i guess , correct me
if i'm wrong )*

when we query curl -i http://
$solrnode:8983/solr/liveaodfuture/suggest?q=Nirvanawt=jsonindent=true

one of the solr node returns
{
  responseHeader:{
status:0,
QTime:0},
  suggest:{
AnalyzingSuggester:{
  Nirvana:{
numFound:1,
suggestions:[{
term:nirvana,
weight:6,
payload:}]}},
DictionarySuggester:{
  Nirvana:{
numFound:0,
suggestions:[]

/admin/luke/collection/ call status

index:{
numDocs:90564,
maxDoc:94583,
deletedDocs:4019,
...}


while other 3 solr node returns

{
  responseHeader:{
status:0,
QTime:1},
  suggest:{
AnalyzingSuggester:{
  Nirvana:{
numFound:2,
suggestions:[{
term:nirvana,
weight:163,
payload:},
*  {*
*term:nirvana cover,*
*weight:11,*
*payload:}]}},*
DictionarySuggester:{
  Nirvana:{
numFound:0,
suggestions:[]

/admin/luke/collection/ call status on other 3 solr nodes... which have
different maxDoc that the above solr node.

index:{
numDocs:90564,
maxDoc:156760,
}

when i check the built time for suggest directory of the collection on each
solr node have the same time

ls -lah /mnt/solrdrive/solr/cores/*/data/suggest_analyzing/*
-rw-r--r-- 1 root root 3.0M May 20 16:00
/mnt/solrdrive/solr/cores/collection1_shard1_replica3/data/suggest_analyzing/wfsta.bin

Questions:
Does the suggester built url i.e,
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true
*consider maxdocs or deleted docs also?
  Does the suggester built from  i.e,
*solr/collection1/suggest?suggest.build=true
*is different from buildOnCommit=true property ?
   Do any one have better solution to keep the suggester current
with contents in the index with more frequent softcommits?

   Does solr have any component like scheduler like cron scheduler
to schedule the suggest build and
 scheduling the optimize on daily basis ?


*Thanks,*
*Rajesh**.*

Re: all terms and frequency

[Rajesh Hazari]

Yes, we can get all the distinct terms from each collection solr index
using the below call

http://{host}:{port}/solr/collection1/admin/luke?wt=jsonfl=fieldnumTerms=2
the fields:{.} of response is your distinct terms in that collection.

I'm sure we can get this using solrj also but never tried.

*Rajesh**.*

On Tue, Jun 2, 2015 at 6:32 AM, Chaushu, Shani shani.chau...@intel.com
wrote:

 Hi,
 There is a way to use the Solr API to see all the words in all the
 documents and their frequency in each document?
 I saw the term vector component but couldn't see any way to see the vector
 for each word...

 Thanks,
 Shani


 -
 Intel Electronics Ltd.

 This e-mail and any attachments may contain confidential material for
 the sole use of the intended recipient(s). Any review or distribution
 by others is strictly prohibited. If you are not the intended
 recipient, please contact the sender and delete all copies.

Re: Best strategy for logging security

[Rajesh Hazari]

Logging :

Just use logstash to a parse your logs for all collection and  logstash
forwarder and lumberjack at your solr replicas in your solr cloud to send
the log events to you central logstash server and send it to back to solr
(either the same or different instance) to a different collection.

The default log4j.properties that comes with solr dist can log core name
with each query log.

Security:
suggest you to go through this wiki
https://wiki.apache.org/solr/SolrSecurity

*Thanks,*
*Rajesh,*
*(mobile) : 8328789519.*

On Mon, Jun 1, 2015 at 11:20 AM, Vishal Swaroop vishal@gmail.com
wrote:

 It will be great if you can provide your valuable inputs on strategy for
 logging  security...


 Thanks a lot in advance...



 Logging :

 - Is there a way to implement logging for each cores separately.

 - What will be the best strategy to log every query details (like source
 IP, search query, etc.) at some point we will need monthly reports for
 analysis.



 Securing SOLR :

 - We need to implement SOLR security from client as well as server side...
 requests will be performed via web app as well as other server side apps
 e.g. curl...

 Please suggest about the best approach we can follow... link to any
 documentation will also help.



 Environment : SOLR 4.7 configured on Tomcat 7  (Linux)

Re: and stopword in user query is being change to q.op=AND

[Rajesh Hazari]

Thanks Show and Hoss.
Just added lowercaseOperators=false to my edismax config and everything
seems to be working.

*Thanks,*
*Rajesh,*
*(mobile) : 8328789519.*

On Mon, Apr 27, 2015 at 11:53 AM, Rajesh Hazari rajeshhaz...@gmail.com
wrote:

 I did go through the documentation of edismax (solr 5.1 documentation),
 that suggests to use *stopwords* query param that signal the parser to
 respect stopfilterfactory while parsing, still i did not find this is
 happening.

 my final query looks like this


 http://host/solr/collection/select?q=term1+and+term2sort=update_time+descrows=1wt=jsonindent=truedebugQuery=truedefType=edismaxstopwords=trugroup=truegroup.ngroups=truegroup.field=titlestopwords=true


 debug:{
 rawquerystring:term1 and term2,
 querystring:term1 and term2,
 parsedquery:(+(+DisjunctionMaxQuery((textSpell:term1)) 
 +DisjunctionMaxQuery((textSpell:term2/no_coord,
 parsedquery_toString:+(+(textSpell:term1) +(textSpell:term2)),
 explain:{},
 QParser:ExtendedDismaxQParser,...

 ..

 Is this param introduced and supports from specific version of solr!

 our solr version is 4.7 and 4.9.


 *Thanks,*
 *Rajesh**.*

 On Sun, Apr 26, 2015 at 9:22 PM, Rajesh Hazari rajeshhaz...@gmail.com
 wrote:

 Thank you Hoss from correcting my understanding, again i missed this
 concept of edismax.

 Do we have any solrj class or helper to handle the scenario to pass on
 the query terms (by stripping stopwords ) to edismax using solrj api.
 for ex: if user queries for *term1 and term2* build and query to pass
 on this to edismax so that this user query will be parsed as


 *parsedquery: (+(DisjunctionMaxQuery((textSpell:term1)
 DisjunctionMaxQuery((textSpell:term2/no_coord *

 *Thanks,*
 *Rajesh**.*

 On Fri, Apr 24, 2015 at 1:13 PM, Chris Hostetter 
 hossman_luc...@fucit.org wrote:


 : I was under understanding that stopwords are filtered even before being
 : parsed by search handler, i do have the filter in collection schema to
 : filter stopwords and the analysis shows that this stopword is filtered

 Generally speaking, your understanding of the order of operations for
 query parsing (regardless of hte parser) and analysis (regardless of the
 fields/analyzers/filters/etc...) is backwards.


 the query parser gets, as it's input, the query string (as a *single*
 string) and the request params.  it inspects/parses the string according
 to it's rules  options  syntax and based on what it finds in that
 string
 (and in other request params) it passes some/all of that string to the
 analyzer for one or more fields, and uses the results of those analyzers
 as the terms for building up a query structure.

 ask yourself: if the raw user query input was first passed to an analyzer
 (for stop word filtering as you suggest) before the being passed to the
 query parser -- how would solr know what analyzer to use?  in many
 parsers
 (like lucene and edismax) the fields to use can be specified *inside* the
 query string itself

 likewise: how would you ensure that syntactically significant string
 sequences (like ( and : and AND etc..) that an analyzer might
 normally strip out based on the tokenizer/tokenfilters would be preserved
 so that the query parser could have them and use them to drive hte
 resulting query structure?



 -Hoss
 http://www.lucidworks.com/

Re: Trying to get AnalyzingInfixSuggester to work in Solr?

[Rajesh Hazari]

Good to know that its working as expected.

I have some couple of question on your autosuggest implementation.

I see that you are using SpellcheckComponent instead of SuggestComponent
are you using this intentionally if not plz read this
 https://cwiki.apache.org/confluence/display/solr/Suggester

I am working on an issue in suggester just sharing once again in this
community just in-case if you are any others out have this in their list.

http://stackoverflow.com/questions/27847707/solr-autosuggest-to-stop-filter-suggesting-the-phrase-that-ends-with-stopwords

*thanks,*
*Rajesh**.*

On Thu, May 7, 2015 at 11:26 AM, O. Olson olson_...@yahoo.it wrote:

 Thank you Erick. I have no clue what you are referring to when you used to
 word this?  Are you referring to my question in my original
 email/message?


 Erick Erickson wrote
  Have you seen this? I tried to make something end-to-end with assorted
  gotchas identified
 
   Best,
  Erick





 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204336.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Trying to get AnalyzingInfixSuggester to work in Solr?

[Rajesh Hazari]

yes textSuggest is of type text_general with below definition

fieldType name=text_general class=solr.TextField
 positionIncrementGap=100 sortMissingLast=true omitNorms=true
 analyzer type=index
tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.ClassicFilterFactory/
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
filter class=solr.ShingleFilterFactory maxShingleSize=5
outputUnigrams=true/
  /analyzer
  analyzer type=query
charFilter class=solr.MappingCharFilterFactory
mapping=mapping-FoldToASCII.txt/
 tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.ClassicFilterFactory/
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
filter class=solr.ShingleFilterFactory maxShingleSize=5
outputUnigrams=true/
  /analyzer
/fieldType

*Rajesh.*

On Wed, May 6, 2015 at 4:50 PM, O. Olson olson_...@yahoo.it wrote:

 Thank you Rajesh for responding so quickly. I tried it again with a restart
 and a reimport and I still cannot get this to work i.e. I'm seeing no
 difference.

 I'm wondering how you define: 'textSuggest' in your schema? In my case I
 use
 the field 'text' that is defined as:

 field name=text type=text_general indexed=true stored=false
 multiValued=true/

 I'm wondering if your 'textSuggest' is of type text_general ?

 Thank you again for your help
 O. O.


 Rajesh Hazari wrote
  I just tested your config with my schema and it worked.
 
  my config :
 
  searchComponent class=solr.SpellCheckComponent name=suggest1
 
  lst name=spellchecker
 
  str name=name
  suggest
  /str
 
  str name=classname
  org.apache.solr.spelling.suggest.Suggester
  /str
 
  str
  name=lookupImpl
  org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory
  /str
 
  str name=field
  textSuggest
  /str
 
  float name=threshold
  0.005
  /float
 
  str name=buildOnCommit
  true
  /str
 
  str name=suggestAnalyzerFieldType
  text_general
  /str
 
  bool name=exactMatchFirst
  true
  /bool
 
  /lst
 
  /searchComponent
 
  queryConverter name=queryConverter
  class=org.apache.solr.spelling.SuggestQueryConverter/
 
  requestHandler class=org.apache.solr.handler.component.SearchHandler
  name=/suggest1
 
  lst name=defaults
 
  str name=spellcheck
  true
  /str
 
  str name=spellcheck.dictionary
  suggest
  /str
 
  str name=spellcheck.onlyMorePopular
  true
  /str
 
  str name=spellcheck.count
  5
  /str
 
  str name=spellcheck.collate
  true
  /str
 
  /lst
 
  arr name=components
 
  str
  suggest1
  /str
 
  /arr
 
  /requestHandler
 
 
 http://localhost:8585/solr/collection1/suggest1?q=applerows=10wt=jsonindent=true
 
  {
responseHeader:{
  status:0,
  QTime:2},
spellcheck:{
  suggestions:[
apple,{
  numFound:5,
  startOffset:0,
  endOffset:5,
  suggestion:[
 *
  apple
 *
  ,

 *
  apple
 *
   and,

 *
  apple
 *
   and facebook,

 *
  apple
 *
   and facebook learn,

 *
  apple
 *
   and facebook learn from]},
collation,
 *
  apple
 *
  ]}}
 
 
 
  *Rajesh**.*





 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204208.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Trying to get AnalyzingInfixSuggester to work in Solr?

[Rajesh Hazari]

Just add the queryConverter definition in your solr config you should use
see multiple term suggestions.
and also make sure you have shingleFilterFactory as one of the filter in
you schema field definitions for your field text_general.

filter class=solr.ShingleFilterFactory maxShingleSize=5
outputUnigrams=true/


*Rajesh**.*

On Wed, May 6, 2015 at 1:47 PM, O. Olson olson_...@yahoo.it wrote:

 Thank you Rajesh. I'm not familiar with the queryConverter. How do you wire
 it up to the rest of the setup? Right now, I just put it between the
 SpellCheckComponent and the RequestHandler i.e. my config is as:

 searchComponent class=solr.SpellCheckComponent name=suggest
 lst name=spellchecker
   str name=namesuggest/str
   str
 name=classnameorg.apache.solr.spelling.suggest.Suggester/str
   str

 name=lookupImplorg.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory/str
   str name=fieldtext/str
   float name=threshold0.005/float
   str name=buildOnCommittrue/str
   str name=suggestAnalyzerFieldTypetext_general/str
   bool name=exactMatchFirsttrue/bool
 /lst
   /searchComponent

   queryConverter name=queryConverter
 class=org.apache.solr.spelling.SuggestQueryConverter/

   requestHandler class=org.apache.solr.handler.component.SearchHandler
 name=/suggest
 lst name=defaults
   str name=spellchecktrue/str
   str name=spellcheck.dictionarysuggest/str
   str name=spellcheck.onlyMorePopulartrue/str
   str name=spellcheck.count5/str
   str name=spellcheck.collatetrue/str
 /lst
 arr name=components
   strsuggest/str
 /arr
   /requestHandler

 Is this correct? I do not see any difference in my results i.e. the
 suggestions are the same as before.
 O. O.





 Rajesh Hazari wrote
  make sure you have this query converter defined in your config
  queryConverter name=queryConverter
  class=org.apache.solr.spelling.SuggestQueryConverter/
  *Thanks,*
  *Rajesh**.*





 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204173.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Trying to get AnalyzingInfixSuggester to work in Solr?

[Rajesh Hazari]

I just tested your config with my schema and it worked.

my config :
  searchComponent class=solr.SpellCheckComponent name=suggest1
lst name=spellchecker
  str name=namesuggest/str
  str name=classnameorg.apache.solr.spelling.suggest.Suggester/str
  str
name=lookupImplorg.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory/str
  str name=fieldtextSuggest/str
  float name=threshold0.005/float
  str name=buildOnCommittrue/str
  str name=suggestAnalyzerFieldTypetext_general/str
  bool name=exactMatchFirsttrue/bool
/lst
  /searchComponent

queryConverter name=queryConverter
class=org.apache.solr.spelling.SuggestQueryConverter/

  requestHandler class=org.apache.solr.handler.component.SearchHandler
name=/suggest1
lst name=defaults
  str name=spellchecktrue/str
  str name=spellcheck.dictionarysuggest/str
  str name=spellcheck.onlyMorePopulartrue/str
  str name=spellcheck.count5/str
  str name=spellcheck.collatetrue/str
/lst
arr name=components
  strsuggest1/str
/arr
  /requestHandler


http://localhost:8585/solr/collection1/suggest1?q=applerows=10wt=jsonindent=true

{
  responseHeader:{
status:0,
QTime:2},
  spellcheck:{
suggestions:[
  apple,{
numFound:5,
startOffset:0,
endOffset:5,
suggestion:[bapple/b,
  bapple/b and,
  bapple/b and facebook,
  bapple/b and facebook learn,
  bapple/b and facebook learn from]},
  collation,bapple/b]}}



*Rajesh**.*

On Wed, May 6, 2015 at 2:48 PM, Rajesh Hazari rajeshhaz...@gmail.com
wrote:

 Just add the queryConverter definition in your solr config you should use
 see multiple term suggestions.
 and also make sure you have shingleFilterFactory as one of the filter in
 you schema field definitions for your field text_general.

 filter class=solr.ShingleFilterFactory maxShingleSize=5
 outputUnigrams=true/


 *Rajesh**.*

 On Wed, May 6, 2015 at 1:47 PM, O. Olson olson_...@yahoo.it wrote:

 Thank you Rajesh. I'm not familiar with the queryConverter. How do you
 wire
 it up to the rest of the setup? Right now, I just put it between the
 SpellCheckComponent and the RequestHandler i.e. my config is as:

 searchComponent class=solr.SpellCheckComponent name=suggest
 lst name=spellchecker
   str name=namesuggest/str
   str
 name=classnameorg.apache.solr.spelling.suggest.Suggester/str
   str

 name=lookupImplorg.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory/str
   str name=fieldtext/str
   float name=threshold0.005/float
   str name=buildOnCommittrue/str
   str name=suggestAnalyzerFieldTypetext_general/str
   bool name=exactMatchFirsttrue/bool
 /lst
   /searchComponent

   queryConverter name=queryConverter
 class=org.apache.solr.spelling.SuggestQueryConverter/

   requestHandler class=org.apache.solr.handler.component.SearchHandler
 name=/suggest
 lst name=defaults
   str name=spellchecktrue/str
   str name=spellcheck.dictionarysuggest/str
   str name=spellcheck.onlyMorePopulartrue/str
   str name=spellcheck.count5/str
   str name=spellcheck.collatetrue/str
 /lst
 arr name=components
   strsuggest/str
 /arr
   /requestHandler

 Is this correct? I do not see any difference in my results i.e.
 the
 suggestions are the same as before.
 O. O.





 Rajesh Hazari wrote
  make sure you have this query converter defined in your config
  queryConverter name=queryConverter
  class=org.apache.solr.spelling.SuggestQueryConverter/
  *Thanks,*
  *Rajesh**.*





 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204173.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Trying to get AnalyzingInfixSuggester to work in Solr?

[Rajesh Hazari]

make sure you have this query converter defined in your config

queryConverter name=queryConverter
class=org.apache.solr.spelling.SuggestQueryConverter/

*Thanks,*
*Rajesh**.*

On Wed, May 6, 2015 at 12:39 PM, O. Olson olson_...@yahoo.it wrote:

 I'm trying to get the AnalyzingInfixSuggester to work but I'm not
 successful.
 I'd be grateful if someone can point me to a working example.

 Problem:
 My content is product descriptions similar to a BestBuy or NewEgg catalog.
 My problem is that I'm getting only single words in the suggester results.
 E.g. if I type 'len', I get the suggester results like 'Lenovo' but not
 'Lenovo laptop' or something larger/longer than a single word.

 There is a suggestion here:

 http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html
 that the search at:
 http://jirasearch.mikemccandless.com/search.py?index=jira is powered by
 the
 AnalyzingInfixSuggester  If this is true, when I use this suggester, I get
 more than a few words in the suggester results, but I don't with my setup
 i.e. on my setup I get only single words. My configuration is


 searchComponent class=solr.SpellCheckComponent name=suggest
 lst name=spellchecker
   str name=namesuggest/str
   str
 name=classnameorg.apache.solr.spelling.suggest.Suggester/str
   str

 name=lookupImplorg.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory/str
   str name=fieldtext/str
   float name=threshold0.005/float
   str name=buildOnCommittrue/str
   str name=suggestAnalyzerFieldTypetext_general/str
   bool name=exactMatchFirsttrue/bool
 /lst
   /searchComponent

   requestHandler class=org.apache.solr.handler.component.SearchHandler
 name=/suggest
 lst name=defaults
   str name=spellchecktrue/str
   str name=spellcheck.dictionarysuggest/str
   str name=spellcheck.onlyMorePopulartrue/str
   str name=spellcheck.count5/str
   str name=spellcheck.collatetrue/str
 /lst
 arr name=components
   strsuggest/str
 /arr
   /requestHandler

 I copy the contents of all of my fields to a single field called 'text'.
 The
 ' text_general' type is exactly as in the solr examples:

 http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/example-DIH/solr/db/conf/schema.xml?view=markup

 I'd be grateful if anyone can help me. I don't know what to look at. Thank
 you in adance.

 O. O.





 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr node going to recovering state during heavy reindexing

[Rajesh Hazari]

our production solr nodes were having similar issue with 4 nodes everything
is normal, but when we try to increase the replicas (nodes) to 10 most of
then went to recovery.
our config params :
nodes : 20 (replica in each node)
soft commit is 6 sec
hard commit is 5 min
indexing scheduled time : every 3 mins around 5k of documents.

Now we are back on 4 nodes in prod, which is working our for this season,
but we may be hitting this case once again in near future where we want to
expand.

I have been going through the blog which suggest soft commit and hard
commit for near real time search instances, may be you can also have a look.

http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/


*Rajesh.*

On Mon, Apr 27, 2015 at 11:15 AM, Gopal Jee gopal@myntra.com wrote:

 We have a 26 node solr cloud cluster. During heavy re-indexing, some of
 nodes go into recovering state.
 as per current config, soft commit is set to 15 minute and hard commit to
 30 sec. Moreover, zkClientTimeout is set to 30 sec in solr nodes.
 Please advise.

 Thanks
 Gopal

Re: and stopword in user query is being change to q.op=AND

[Rajesh Hazari]

I did go through the documentation of edismax (solr 5.1 documentation),
that suggests to use *stopwords* query param that signal the parser to
respect stopfilterfactory while parsing, still i did not find this is
happening.

my final query looks like this

http://host/solr/collection/select?q=term1+and+term2sort=update_time+descrows=1wt=jsonindent=truedebugQuery=truedefType=edismaxstopwords=trugroup=truegroup.ngroups=truegroup.field=titlestopwords=true


debug:{
rawquerystring:term1 and term2,
querystring:term1 and term2,
parsedquery:(+(+DisjunctionMaxQuery((textSpell:term1))
+DisjunctionMaxQuery((textSpell:term2/no_coord,
parsedquery_toString:+(+(textSpell:term1) +(textSpell:term2)),
explain:{},
QParser:ExtendedDismaxQParser,...

..

Is this param introduced and supports from specific version of solr!

our solr version is 4.7 and 4.9.


*Thanks,*
*Rajesh**.*

On Sun, Apr 26, 2015 at 9:22 PM, Rajesh Hazari rajeshhaz...@gmail.com
wrote:

 Thank you Hoss from correcting my understanding, again i missed this
 concept of edismax.

 Do we have any solrj class or helper to handle the scenario to pass on the
 query terms (by stripping stopwords ) to edismax using solrj api.
 for ex: if user queries for *term1 and term2* build and query to pass
 on this to edismax so that this user query will be parsed as


 *parsedquery: (+(DisjunctionMaxQuery((textSpell:term1)
 DisjunctionMaxQuery((textSpell:term2/no_coord *

 *Thanks,*
 *Rajesh**.*

 On Fri, Apr 24, 2015 at 1:13 PM, Chris Hostetter hossman_luc...@fucit.org
  wrote:


 : I was under understanding that stopwords are filtered even before being
 : parsed by search handler, i do have the filter in collection schema to
 : filter stopwords and the analysis shows that this stopword is filtered

 Generally speaking, your understanding of the order of operations for
 query parsing (regardless of hte parser) and analysis (regardless of the
 fields/analyzers/filters/etc...) is backwards.


 the query parser gets, as it's input, the query string (as a *single*
 string) and the request params.  it inspects/parses the string according
 to it's rules  options  syntax and based on what it finds in that string
 (and in other request params) it passes some/all of that string to the
 analyzer for one or more fields, and uses the results of those analyzers
 as the terms for building up a query structure.

 ask yourself: if the raw user query input was first passed to an analyzer
 (for stop word filtering as you suggest) before the being passed to the
 query parser -- how would solr know what analyzer to use?  in many parsers
 (like lucene and edismax) the fields to use can be specified *inside* the
 query string itself

 likewise: how would you ensure that syntactically significant string
 sequences (like ( and : and AND etc..) that an analyzer might
 normally strip out based on the tokenizer/tokenfilters would be preserved
 so that the query parser could have them and use them to drive hte
 resulting query structure?



 -Hoss
 http://www.lucidworks.com/

Re: Solr node going to recovering state during heavy reindexing

[Rajesh Hazari]

thanks, i am sure that we have missed this command line property, this
gives me more information on how to use latest solr scripts more
effectively.


*Thanks,*
*Rajesh**.*

On Mon, Apr 27, 2015 at 12:04 PM, Shawn Heisey apa...@elyograg.org wrote:

 On 4/27/2015 9:15 AM, Gopal Jee wrote:
  We have a 26 node solr cloud cluster. During heavy re-indexing, some of
  nodes go into recovering state.
  as per current config, soft commit is set to 15 minute and hard commit to
  30 sec. Moreover, zkClientTimeout is set to 30 sec in solr nodes.
  Please advise.

 The most common reason for this is general performance issues that make
 some operations take longer than the zkClientTimeout.

 My first suspect would be long garbage collection pauses.  This assumes
 you're not using a very recent version (4.10.x or 5.x) with the new
 bin/solr script, and your java commandline does not have any garbage
 collection tuning.  The bin/solr script does a lot of GC tuning.

 The second suspect would be that you don't have enough RAM left for your
 operating system to cache your index effectively.

 It's possible to have both of these problems happening.  These problems,
 and a few others, are outlined here:

 http://wiki.apache.org/solr/SolrPerformanceProblems

 Thanks,
 Shawn

Re: and stopword in user query is being change to q.op=AND

[Rajesh Hazari]

Thank you Hoss from correcting my understanding, again i missed this
concept of edismax.

Do we have any solrj class or helper to handle the scenario to pass on the
query terms (by stripping stopwords ) to edismax using solrj api.
for ex: if user queries for *term1 and term2* build and query to pass on
this to edismax so that this user query will be parsed as


*parsedquery: (+(DisjunctionMaxQuery((textSpell:term1)
DisjunctionMaxQuery((textSpell:term2/no_coord *

*Thanks,*
*Rajesh**.*

On Fri, Apr 24, 2015 at 1:13 PM, Chris Hostetter hossman_luc...@fucit.org
wrote:


 : I was under understanding that stopwords are filtered even before being
 : parsed by search handler, i do have the filter in collection schema to
 : filter stopwords and the analysis shows that this stopword is filtered

 Generally speaking, your understanding of the order of operations for
 query parsing (regardless of hte parser) and analysis (regardless of the
 fields/analyzers/filters/etc...) is backwards.


 the query parser gets, as it's input, the query string (as a *single*
 string) and the request params.  it inspects/parses the string according
 to it's rules  options  syntax and based on what it finds in that string
 (and in other request params) it passes some/all of that string to the
 analyzer for one or more fields, and uses the results of those analyzers
 as the terms for building up a query structure.

 ask yourself: if the raw user query input was first passed to an analyzer
 (for stop word filtering as you suggest) before the being passed to the
 query parser -- how would solr know what analyzer to use?  in many parsers
 (like lucene and edismax) the fields to use can be specified *inside* the
 query string itself

 likewise: how would you ensure that syntactically significant string
 sequences (like ( and : and AND etc..) that an analyzer might
 normally strip out based on the tokenizer/tokenfilters would be preserved
 so that the query parser could have them and use them to drive hte
 resulting query structure?



 -Hoss
 http://www.lucidworks.com/

Re: and stopword in user query is being change to q.op=AND

[Rajesh Hazari]

I was under understanding that stopwords are filtered even before being
parsed by search handler, i do have the filter in collection schema to
filter stopwords and the analysis shows that this stopword is filtered

Analysis response :  attached is the solr analysis json response.

[image: Inline image 1]
Schema definition :
fieldType name=text_general_rev class=solr.TextField
positionIncrementGap=100
analyzer type=index tokenizer class=solr.StandardTokenizerFactory/ 
filter class=solr.StopFilterFactory ignoreCase=true words=
stopwords.txt / filter class=solr.LowerCaseFilterFactory/ filter
class=solr.ReversedWildcardFilterFactory withOriginal=true
maxPosAsterisk=3 maxPosQuestion=2 maxFractionAsterisk=0.33/ /
analyzer analyzer type=query tokenizer class=
solr.StandardTokenizerFactory/ filter class=solr.SynonymFilterFactory
synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=
solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter
class=solr.LowerCaseFilterFactory/ /analyzer /fieldType
field name=textSpell type=text_general indexed=true stored=false
multiValued=true/


*shouldn't the final filter query terms be sent to search handler?*

*Thanks,*
*Rajesh**.*

On Thu, Apr 23, 2015 at 2:56 PM, Chris Hostetter hossman_luc...@fucit.org
wrote:


 : And stopword  in user query is being changed to q.op=AND, i am going to
 : look more into this

 This is an explicitly documented feature of the edismax parser...


 https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser

 * treats and and or as AND and OR in Lucene syntax mode.

 ...

 The lowercaseOperators Parameter

 A Boolean parameter indicating if lowercase and and or should be
 treated the same as operators AND and OR.




 : i thought of sharing this in solr community just in-case if someone have
 : came across this issue.
 : OR
 : I will also be validating my config and schema if i am doing something
 : wrong.
 :
 : solr : 4.9
 : query parser: edismax
 :
 : when i search for *q=derek and romace* final parsed query is
 : *(+(+DisjunctionMaxQuery((textSpell:derek))
 : +DisjunctionMaxQuery((textSpell:romance/no_coord *
 : *
 :   response:{numFound:0,start:0,maxScore:0.0,docs:[]*
 :
 : when i search for *q=derek romace* final parsed query is
 *parsedquery:
 : (+(DisjunctionMaxQuery((textSpell:derek))
 : DisjunctionMaxQuery((textSpell:romance/no_coord,*
 : *response: {*
 : *numFound: 1405,*
 : *start: 0,*
 : *maxScore: 0.2780709,*
 : *docs: [.*
 :
 : textSpell field definition :
 :
 : field name=textSpell type=text_general indexed=true stored=false
 : omitNorms=true multiValued=true /
 :
 : fieldType name=text_general class=solr.TextField
 : positionIncrementGap=100
 :   analyzer type=index
 : tokenizer class=solr.ClassicTokenizerFactory/
 : filter class=solr.StopFilterFactory ignoreCase=true
 : words=stopwords.txt /
 : filter class=solr.LowerCaseFilterFactory/
 : filter class=solr.KeywordMarkerFilterFactory
 : protected=protwords.txt/
 :   /analyzer
 :   analyzer type=query
 : tokenizer class=solr.ClassicTokenizerFactory/
 : filter class=solr.StopFilterFactory ignoreCase=true
 : words=stopwords.txt /
 : filter class=solr.SynonymFilterFactory synonyms=synonyms.txt
 : ignoreCase=true expand=false  /
 : filter class=solr.LowerCaseFilterFactory/
 : filter class=solr.KeywordMarkerFilterFactory
 : protected=protwords.txt/
 :   /analyzer
 : /fieldType
 :
 : Let me know if anyone of you guys need more info.
 :
 : *Thanks,*
 : *Rajesh**.*
 :

 -Hoss
 http://www.lucidworks.com/



analysis.json
Description: application/json

and stopword in user query is being change to q.op=AND

[Rajesh Hazari]

Hi All,

And stopword  in user query is being changed to q.op=AND, i am going to
look more into this
i thought of sharing this in solr community just in-case if someone have
came across this issue.
OR
I will also be validating my config and schema if i am doing something
wrong.

solr : 4.9
query parser: edismax

when i search for *q=derek and romace* final parsed query is
*(+(+DisjunctionMaxQuery((textSpell:derek))
+DisjunctionMaxQuery((textSpell:romance/no_coord *
*
  response:{numFound:0,start:0,maxScore:0.0,docs:[]*

when i search for *q=derek romace* final parsed query is *parsedquery:
(+(DisjunctionMaxQuery((textSpell:derek))
DisjunctionMaxQuery((textSpell:romance/no_coord,*
*response: {*
*numFound: 1405,*
*start: 0,*
*maxScore: 0.2780709,*
*docs: [.*

textSpell field definition :

field name=textSpell type=text_general indexed=true stored=false
omitNorms=true multiValued=true /

fieldType name=text_general class=solr.TextField
positionIncrementGap=100
  analyzer type=index
tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.StopFilterFactory ignoreCase=true
words=stopwords.txt /
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
  /analyzer
  analyzer type=query
tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.StopFilterFactory ignoreCase=true
words=stopwords.txt /
filter class=solr.SynonymFilterFactory synonyms=synonyms.txt
ignoreCase=true expand=false  /
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
  /analyzer
/fieldType

Let me know if anyone of you guys need more info.

*Thanks,*
*Rajesh**.*

Re: How to start solr in solr cloud mode using external zookeeper ?

[Rajesh Hazari]

zkhost=hostnames,
port=some port
variables in your solr.xml should work?
I have tested this with tomcat not with jetty, this stays with your config.

Rajesh.
On Mar 5, 2015 9:20 PM, Aman Tandon amantandon...@gmail.com wrote:

 Thanks shamik :)

 With Regards
 Aman Tandon

 On Fri, Mar 6, 2015 at 3:30 AM, shamik sham...@gmail.com wrote:

  The other way you can do that is to specify the startup parameters in
  solr.in.sh.
 
  Example :
 
  SOLR_MODE=solrcloud
 
  ZK_HOST=zoohost1:2181,zoohost2:2181,zoohost3:2181
 
  SOLR_PORT=4567
 
  You can simply start solr by running ./solr start
 
 
 
  --
  View this message in context:
 
 http://lucene.472066.n3.nabble.com/How-to-start-solr-in-solr-cloud-mode-using-external-zookeeper-tp4190630p4191286.html
  Sent from the Solr - User mailing list archive at Nabble.com.
 

Re: Solr logs encoding

[Rajesh Hazari]

I have seen this log earlier,  I just changed the log level of this class
to WARN.
On Feb 27, 2015 12:03 AM, Moshe Recanati mos...@kmslh.com wrote:

  Hi,

 I've wired situation. Starting yesterday restart I've issue with log
 encoding. My log looks like:

 DEBUG - 2015-02-27 10:47:01.432; 
 [0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xc7]8[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x89][0x5][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x97][0x4][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xa4][0x6][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfc]b[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfc]F[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfb]:[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]a[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]v[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]Y[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]Y[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]V[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]H[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]U[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]\[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xe4][0x96][0x1][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]`[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]j[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]l[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]j[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]X[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]e[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xdd][0xba][0x1][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]h[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xb5][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xee][0x3][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]\[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xe2][0x1d][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xbb][0x1a][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]c[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xd2]%[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]b[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x92][0x1a][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xa3][0x4][0x4][0xfc][0xff][0xff][0xff]



 Anyone familiar with this? How to fix it?





 *Regards,*

 *Moshe Recanati*

 *SVP Engineering*

 Office + 972-73-2617564

 Mobile  + 972-52-6194481

 Skype:  recanati
 [image: KMS2]
 http://finance.yahoo.com/news/kms-lighthouse-named-gartner-cool-121000184.html

 More at:  www.kmslh.com | LinkedIn
 http://www.linkedin.com/company/kms-lighthouse | FB
 https://www.facebook.com/pages/KMS-lighthouse/123774257810917

Re: Collations are not working fine.

[Rajesh Hazari]

Below is the filed definition that we used its just a basic definition ::

analyzer type=index
tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.StopFilterFactory ignoreCase=true
words=stopwords.txt /
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/
  /analyzer
  analyzer type=query
tokenizer class=solr.ClassicTokenizerFactory/
filter class=solr.StopFilterFactory ignoreCase=true
words=stopwords.txt /
filter class=solr.SynonymFilterFactory synonyms=synonyms.txt
ignoreCase=true expand=false  /
filter class=solr.LowerCaseFilterFactory/
filter class=solr.KeywordMarkerFilterFactory
protected=protwords.txt/

  /analyzer




*Rajesh.*


On Thu, Feb 26, 2015 at 2:03 AM, Nitin Solanki nitinml...@gmail.com wrote:

 Hi Rajesh,
 What configuration had you set in your schema.xml?

 On Sat, Feb 14, 2015 at 2:18 AM, Rajesh Hazari rajeshhaz...@gmail.com
 wrote:

  Hi Nitin,
 
  Can u try with the below config, we have these config seems to be working
  for us.
 
  searchComponent name=spellcheck class=solr.SpellCheckComponent
 
   str name=queryAnalyzerFieldTypetext_general/str
 
 
lst name=spellchecker
  str name=namewordbreak/str
  str name=classnamesolr.WordBreakSolrSpellChecker/str
  str name=fieldtextSpell/str
  str name=combineWordstrue/str
  str name=breakWordsfalse/str
  int name=maxChanges5/int
/lst
 
 lst name=spellchecker
  str name=namedefault/str
  str name=fieldtextSpell/str
  str name=classnamesolr.IndexBasedSpellChecker/str
  str name=spellcheckIndexDir./spellchecker/str
  str name=accuracy0.75/str
  float name=thresholdTokenFrequency0.01/float
  str name=buildOnCommittrue/str
  str name=spellcheck.maxResultsForSuggest5/str
   /lst
 
 
/searchComponent
 
 
 
  str name=spellchecktrue/str
  str name=spellcheck.dictionarydefault/str
  str name=spellcheck.dictionarywordbreak/str
  int name=spellcheck.count5/int
  str name=spellcheck.alternativeTermCount15/str
  str name=spellcheck.collatetrue/str
  str name=spellcheck.onlyMorePopularfalse/str
  str name=spellcheck.extendedResultstrue/str
  str name =spellcheck.maxCollations100/str
  str name=spellcheck.collateParam.mm100%/str
  str name=spellcheck.collateParam.q.opAND/str
  str name=spellcheck.maxCollationTries1000/str
 
 
  *Rajesh.*
 
  On Fri, Feb 13, 2015 at 1:01 PM, Dyer, James 
 james.d...@ingramcontent.com
  
  wrote:
 
   Nitin,
  
   Can you post the full spellcheck response when you query:
  
   q=gram_ci:gone wthh thes wintwt=jsonindent=trueshards.qt=/spell
  
   James Dyer
   Ingram Content Group
  
  
   -Original Message-
   From: Nitin Solanki [mailto:nitinml...@gmail.com]
   Sent: Friday, February 13, 2015 1:05 AM
   To: solr-user@lucene.apache.org
   Subject: Re: Collations are not working fine.
  
   Hi James Dyer,
 I did the same as you told me. Used
   WordBreakSolrSpellChecker instead of shingles. But still collations are
  not
   coming or working.
   For instance, I tried to get collation of gone with the wind by
  searching
   gone wthh thes wint on field=gram_ci but didn't succeed. Even, I am
   getting the suggestions of wtth as *with*, thes as *the*, wint as
 *wind*.
   Also I have documents which contains gone with the wind having 167
  times
   in the documents. I don't know that I am missing something or not.
   Please check my below solr configuration:
  
   *URL: *localhost:8983/solr/wikingram/spell?q=gram_ci:gone wthh thes
   wintwt=jsonindent=trueshards.qt=/spell
  
   *solrconfig.xml:*
  
   searchComponent name=spellcheck class=solr.SpellCheckComponent
   str name=queryAnalyzerFieldTypetextSpellCi/str
   lst name=spellchecker
 str name=namedefault/str
 str name=fieldgram_ci/str
 str name=classnamesolr.DirectSolrSpellChecker/str
 str name=distanceMeasureinternal/str
 float name=accuracy0.5/float
 int name=maxEdits2/int
 int name=minPrefix0/int
 int name=maxInspections5/int
 int name=minQueryLength2/int
 float name=maxQueryFrequency0.9/float
 str name=comparatorClassfreq/str
   /lst
   lst name=spellchecker
 str name=namewordbreak/str
 str name=classnamesolr.WordBreakSolrSpellChecker/str
 str name=fieldgram/str
 str name=combineWordstrue/str
 str name=breakWordstrue/str
 int name=maxChanges5/int
   /lst
   /searchComponent
  
   requestHandler name=/spell class=solr.SearchHandler
 startup=lazy
   lst name=defaults
 str name=dfgram_ci/str
 str name=spellcheck.dictionarydefault/str
 str name=spellcheckon/str
 str name=spellcheck.extendedResultstrue/str
 str name=spellcheck.count25/str
 str name=spellcheck.onlyMorePopulartrue/str
 str name=spellcheck.maxResultsForSuggest1/str

Re: Collations are not working fine.

[Rajesh Hazari]

 Suggestions (spellcheck)/str
  str name=echoParamsexplicit/str
  str name=wtjson/str
  str name=rows0/str
  str name=defTypeedismax/str
  str name=dftext_all/str
  str
  name=flid,name,ticker,entityType,transactionType,accountType/str
  str name=spellchecktrue/str
  str name=spellcheck.count5/str
  str name=spellcheck.dictionarysuggestDictionary/str
  str name=spellcheck.alternativeTermCount5/str
  str name=spellcheck.collatetrue/str
  str name=spellcheck.extendedResultstrue/str
  str name=spellcheck.maxCollationTries10/str
  str name=spellcheck.maxCollations5/str
/lst
arr name=last-components
  strsuggestSC/str
/arr
  /requestHandler
 
  -Original Message-
  From: Nitin Solanki [mailto:nitinml...@gmail.com]
  Sent: Tuesday, February 17, 2015 3:17 AM
  To: solr-user@lucene.apache.org
  Subject: Re: Collations are not working fine.
 
  Hi Charles,
   Will you please send the configuration which you tried.
  It will help to solve my problem. Have you sorted the collations on hits
 or
  frequencies of suggestions? If you did than please assist me.
 
  On Mon, Feb 16, 2015 at 7:59 PM, Reitzel, Charles 
  charles.reit...@tiaa-cref.org wrote:
 
   I have been working with collations the last couple days and I kept
  adding
   the collation-related parameters until it started working for me.   It
   seems I needed str name=spellcheck.collateMaxCollectDocs50/str.
  
   But, I am using the Suggester with the WFSTLookupFactory.
  
   Also, I needed to patch the suggester to get frequency information in
   the spellcheck response.
  
   -Original Message-
   From: Rajesh Hazari [mailto:rajeshhaz...@gmail.com]
   Sent: Friday, February 13, 2015 3:48 PM
   To: solr-user@lucene.apache.org
   Subject: Re: Collations are not working fine.
  
   Hi Nitin,
  
   Can u try with the below config, we have these config seems to be
   working for us.
  
   searchComponent name=spellcheck class=solr.SpellCheckComponent
  
str name=queryAnalyzerFieldTypetext_general/str
  
  
 lst name=spellchecker
   str name=namewordbreak/str
   str name=classnamesolr.WordBreakSolrSpellChecker/str
   str name=fieldtextSpell/str
   str name=combineWordstrue/str
   str name=breakWordsfalse/str
   int name=maxChanges5/int
 /lst
  
  lst name=spellchecker
   str name=namedefault/str
   str name=fieldtextSpell/str
   str name=classnamesolr.IndexBasedSpellChecker/str
   str name=spellcheckIndexDir./spellchecker/str
   str name=accuracy0.75/str
   float name=thresholdTokenFrequency0.01/float
   str name=buildOnCommittrue/str
   str name=spellcheck.maxResultsForSuggest5/str
/lst
  
  
 /searchComponent
  
  
  
   str name=spellchecktrue/str
   str name=spellcheck.dictionarydefault/str
   str name=spellcheck.dictionarywordbreak/str
   int name=spellcheck.count5/int
   str name=spellcheck.alternativeTermCount15/str
   str name=spellcheck.collatetrue/str
   str name=spellcheck.onlyMorePopularfalse/str
   str name=spellcheck.extendedResultstrue/str
   str name =spellcheck.maxCollations100/str
   str name=spellcheck.collateParam.mm100%/str
   str name=spellcheck.collateParam.q.opAND/str
   str name=spellcheck.maxCollationTries1000/str
  
  
   *Rajesh.*
  
   On Fri, Feb 13, 2015 at 1:01 PM, Dyer, James
   james.d...@ingramcontent.com
   
   wrote:
  
Nitin,
   
Can you post the full spellcheck response when you query:
   
q=gram_ci:gone wthh thes wintwt=jsonindent=trueshards.qt=/spell
   
James Dyer
Ingram Content Group
   
   
-Original Message-
From: Nitin Solanki [mailto:nitinml...@gmail.com]
Sent: Friday, February 13, 2015 1:05 AM
To: solr-user@lucene.apache.org
Subject: Re: Collations are not working fine.
   
Hi James Dyer,
  I did the same as you told me. Used
WordBreakSolrSpellChecker instead of shingles. But still collations
are not coming or working.
For instance, I tried to get collation of gone with the wind by
searching gone wthh thes wint on field=gram_ci but didn't succeed.
Even, I am getting the suggestions of wtth as *with*, thes as *the*,
   wint as *wind*.
Also I have documents which contains gone with the wind having 167
times in the documents. I don't know that I am missing something or
  not.
Please check my below solr configuration:
   
*URL: *localhost:8983/solr/wikingram/spell?q=gram_ci:gone wthh thes
wintwt=jsonindent=trueshards.qt=/spell
   
*solrconfig.xml:*
   
searchComponent name=spellcheck class=solr.SpellCheckComponent
str name=queryAnalyzerFieldTypetextSpellCi/str
lst name=spellchecker
  str name=namedefault/str
  str name=fieldgram_ci/str
  str name=classnamesolr.DirectSolrSpellChecker/str
  str name=distanceMeasureinternal/str
  float name=accuracy0.5/float
  int name=maxEdits2/int
  int name

Re: Collations are not working fine.

[Rajesh Hazari]

Hi Nitin,

Can u try with the below config, we have these config seems to be working
for us.

searchComponent name=spellcheck class=solr.SpellCheckComponent

 str name=queryAnalyzerFieldTypetext_general/str


  lst name=spellchecker
str name=namewordbreak/str
str name=classnamesolr.WordBreakSolrSpellChecker/str
str name=fieldtextSpell/str
str name=combineWordstrue/str
str name=breakWordsfalse/str
int name=maxChanges5/int
  /lst

   lst name=spellchecker
str name=namedefault/str
str name=fieldtextSpell/str
str name=classnamesolr.IndexBasedSpellChecker/str
str name=spellcheckIndexDir./spellchecker/str
str name=accuracy0.75/str
float name=thresholdTokenFrequency0.01/float
str name=buildOnCommittrue/str
str name=spellcheck.maxResultsForSuggest5/str
 /lst


  /searchComponent



str name=spellchecktrue/str
str name=spellcheck.dictionarydefault/str
str name=spellcheck.dictionarywordbreak/str
int name=spellcheck.count5/int
str name=spellcheck.alternativeTermCount15/str
str name=spellcheck.collatetrue/str
str name=spellcheck.onlyMorePopularfalse/str
str name=spellcheck.extendedResultstrue/str
str name =spellcheck.maxCollations100/str
str name=spellcheck.collateParam.mm100%/str
str name=spellcheck.collateParam.q.opAND/str
str name=spellcheck.maxCollationTries1000/str


*Rajesh.*

On Fri, Feb 13, 2015 at 1:01 PM, Dyer, James james.d...@ingramcontent.com
wrote:

 Nitin,

 Can you post the full spellcheck response when you query:

 q=gram_ci:gone wthh thes wintwt=jsonindent=trueshards.qt=/spell

 James Dyer
 Ingram Content Group


 -Original Message-
 From: Nitin Solanki [mailto:nitinml...@gmail.com]
 Sent: Friday, February 13, 2015 1:05 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Collations are not working fine.

 Hi James Dyer,
   I did the same as you told me. Used
 WordBreakSolrSpellChecker instead of shingles. But still collations are not
 coming or working.
 For instance, I tried to get collation of gone with the wind by searching
 gone wthh thes wint on field=gram_ci but didn't succeed. Even, I am
 getting the suggestions of wtth as *with*, thes as *the*, wint as *wind*.
 Also I have documents which contains gone with the wind having 167 times
 in the documents. I don't know that I am missing something or not.
 Please check my below solr configuration:

 *URL: *localhost:8983/solr/wikingram/spell?q=gram_ci:gone wthh thes
 wintwt=jsonindent=trueshards.qt=/spell

 *solrconfig.xml:*

 searchComponent name=spellcheck class=solr.SpellCheckComponent
 str name=queryAnalyzerFieldTypetextSpellCi/str
 lst name=spellchecker
   str name=namedefault/str
   str name=fieldgram_ci/str
   str name=classnamesolr.DirectSolrSpellChecker/str
   str name=distanceMeasureinternal/str
   float name=accuracy0.5/float
   int name=maxEdits2/int
   int name=minPrefix0/int
   int name=maxInspections5/int
   int name=minQueryLength2/int
   float name=maxQueryFrequency0.9/float
   str name=comparatorClassfreq/str
 /lst
 lst name=spellchecker
   str name=namewordbreak/str
   str name=classnamesolr.WordBreakSolrSpellChecker/str
   str name=fieldgram/str
   str name=combineWordstrue/str
   str name=breakWordstrue/str
   int name=maxChanges5/int
 /lst
 /searchComponent

 requestHandler name=/spell class=solr.SearchHandler startup=lazy
 lst name=defaults
   str name=dfgram_ci/str
   str name=spellcheck.dictionarydefault/str
   str name=spellcheckon/str
   str name=spellcheck.extendedResultstrue/str
   str name=spellcheck.count25/str
   str name=spellcheck.onlyMorePopulartrue/str
   str name=spellcheck.maxResultsForSuggest1/str
   str name=spellcheck.alternativeTermCount25/str
   str name=spellcheck.collatetrue/str
   str name=spellcheck.maxCollations50/str
   str name=spellcheck.maxCollationTries50/str
   str name=spellcheck.collateExtendedResultstrue/str
 /lst
 arr name=last-components
   strspellcheck/str
 /arr
   /requestHandler

 *Schema.xml: *

 field name=gram_ci type=textSpellCi indexed=true stored=true
 multiValued=false/

 /fieldTypefieldType name=textSpellCi class=solr.TextField
 positionIncrementGap=100
analyzer type=index
 tokenizer class=solr.StandardTokenizerFactory/
 filter class=solr.LowerCaseFilterFactory/
 /analyzer
 analyzer type=query
 tokenizer class=solr.StandardTokenizerFactory/
 filter class=solr.LowerCaseFilterFactory/
 /analyzer
 /fieldType

Re: Trending functionality in Solr

[Rajesh Hazari]

Hi folks,

In the project that i am working now, we have used logstash to parse the
log files and store the user queries back into a separate collection in
solr and banana dashboard configure to view the usage.

logstash: logstash.net/
banana : github.com/LucidWorks/banana

*Rajesh.*


On Mon, Feb 9, 2015 at 5:04 AM, Alvaro Cabrerizo topor...@gmail.com wrote:

 Hi,

 In a project I'm working on, we adopted the same approach as the one
 commented by Jorge Luis (use a separate core and let the front-end send the
 query to solr to store it). Depending on the analysis you want to achieve
 it could be interesting to delete duplicates or not (have 1 solr document
 per reqeuest). Apart from that, in order to inject all the legacy data we
 used logstash http://logstash.net/ (for log parsing) and banana
 https://github.com/LucidWorks/banana (for reporting).

 Regards.


 On Mon, Feb 9, 2015 at 9:14 AM, Siegfried Goeschl sgoes...@gmx.at wrote:

  Hi folks,
 
  I implemented something similar but never got around to contribute it -
  see https://issues.apache.org/jira/browse/SOLR-4056
 
  The code was initially for SOLR3 but was recently ported to SOLR4
 
  * capturing the most frequent search terms per core
  * supports ad-hoc queries
  * CSV export
 
  If you are interested we could team up and make a proper SOLR
 contribution
  :-)
 
  Cheers,
 
  Siegfried Goeschl
 
 
 
  On 08.02.15 05:26, S.L wrote:
 
  Folks,
 
  Is there a way to implement the trending functionality using Solr , to
  give
  the results using a query for say the most searched terms in the past
  hours
  or so , if the most searched terms is not possible is it possible to at
  least the get results for the last 100 terms?
 
  Thanks
 
 
 

Advantages over solrj api over Spring Data Solr api

[Rajesh Hazari]

Hi folks,

We are using solrj api at large to index and query, wondering if any one
have already used any version of spring data solr and what are the
advantages over solrj api?

We are exploring other new technologies or updates around solr.

*Rajesh.*

Re: shell script or script in any language to scale a replica solr node with some configs from zookeeper and the remaining from svn/git

[Rajesh Hazari]

we have already started using this toolkit, we have explored it completely,

Do we have any sample script in python to get the config file or other
files from svn and deploy in tomcat?


*Thanks,*
*Rajesh**.*

On Mon, Feb 2, 2015 at 3:32 PM, Anshum Gupta ans...@anshumgupta.net wrote:

 Solr scale toolkit should be a good option for you when it comes to
 deploying/managing Solr nodes in a cluster.
 It has a lot of support for stuff like spinning up new nodes, stopping,
 patching, rolling restart etc.

 About not knowing python, as is mentioned in the README, you don't really
 need to know python to be able to use the toolkit.

 On Mon, Feb 2, 2015 at 12:17 PM, Rajesh Hazari rajeshhaz...@gmail.com
 wrote:

  Hi Folks,
 
  Can one one you share the shell script or any script in other language to
  spin up a new solr node deployed in tomcat
  with most of the configs from zookeepers and some from svn, with some
  defaults values.
 
  #some default directory
  ${solrDataDir} =/opt
  #some host name
  ${masterHost}=dev.solr.node1
  solr.home= ${solrDataDir}/solr/
  #default log directory
  solr.log=${solrDataDir}/solr/logs/
  solr.master.url=http://${masterHost}/solr
  tomcat.home=${solrDataDir}/tomcat7/
 
  Get the list of all collections from http://
  {host}/solr/admin/collections?action=CLUSTERSTATUSwt=xml
  and the for each collection
  create its corresponding core directories and auto create core.properties
  for ex: for collection1 and collection2 from above admin response
  sudo mkdir solr.home/collection1
  sudo touch solr.home/collection1/core/properties and populate with
 required
  properties
 
  sudo mkdir solr.home/collection2
  sudo touch solr.home/collection2/core/properties and populate with
 required
  properties
 
  Create a solr.xml file with
  zkhost, port, context and other required properties.
  sudo touch solr.home/solr.xml and populate with required properties
 
  Deploy the solr.war from local svn/any repository to
 ${tomcat.home}/webapps
  dir and start, solr instance started will join the solr cloud farm and
 the
  index will be replicated.
 
  As of now i have two link that i am reading, these scripts are from
  lucidworks i am not sure if we can use them
  and more over these usese phyton and fabric which are very new to me as
 we
  do not have any experienced developer with these skills
 
  https://github.com/LucidWorks/solr-scale-tk
and
  http://lucidworks.com/blog/introducing-the-solr-scale-toolkit/
 
 
  *Thanks,*
  *Rajesh.*
 



 --
 Anshum Gupta
 http://about.me/anshumgupta

shell script or script in any language to scale a replica solr node with some configs from zookeeper and the remaining from svn/git

[Rajesh Hazari]

Hi Folks,

Can one one you share the shell script or any script in other language to
spin up a new solr node deployed in tomcat
with most of the configs from zookeepers and some from svn, with some
defaults values.

#some default directory
${solrDataDir} =/opt
#some host name
${masterHost}=dev.solr.node1
solr.home= ${solrDataDir}/solr/
#default log directory
solr.log=${solrDataDir}/solr/logs/
solr.master.url=http://${masterHost}/solr
tomcat.home=${solrDataDir}/tomcat7/

Get the list of all collections from http://
{host}/solr/admin/collections?action=CLUSTERSTATUSwt=xml
and the for each collection
create its corresponding core directories and auto create core.properties
for ex: for collection1 and collection2 from above admin response
sudo mkdir solr.home/collection1
sudo touch solr.home/collection1/core/properties and populate with required
properties

sudo mkdir solr.home/collection2
sudo touch solr.home/collection2/core/properties and populate with required
properties

Create a solr.xml file with
zkhost, port, context and other required properties.
sudo touch solr.home/solr.xml and populate with required properties

Deploy the solr.war from local svn/any repository to ${tomcat.home}/webapps
dir and start, solr instance started will join the solr cloud farm and the
index will be replicated.

As of now i have two link that i am reading, these scripts are from
lucidworks i am not sure if we can use them
and more over these usese phyton and fabric which are very new to me as we
do not have any experienced developer with these skills

https://github.com/LucidWorks/solr-scale-tk
  and
http://lucidworks.com/blog/introducing-the-solr-scale-toolkit/


*Thanks,*
*Rajesh.*

Re: permanently reducing logging levels for Solr

[Rajesh Hazari]

Hi,

Just add log4j.logger.org.apache.solr=SEVERE to you log4j properties.

*Thanks,*
*Rajesh,*
*(mobile) : 8328789519.*

On Wed, Jan 21, 2015 at 3:14 PM, Nemani, Raj raj.nem...@turner.com wrote:

 All,

 How can I reduce the logging levels to SEVERE that survives a Tomcat
 restart or a machine reboot in Solr.  As you may know, I can change the
 logging levels from the logging page in admin console but those changes are
 not persistent across Tomcat server restart or machine reboot.
 Following is the information about the Solr version from Info page in
 admin console.

 Solr Specification Version: 3.2.0
 Solr Implementation Version: 3.2.0 1129474 - rmuir - 2011-05-30 23:07:15
 Lucene Specification Version: 3.2.0
 Lucene Implementation Version: 3.2.0 1129474 - 2011-05-30 23:08:57

 Please let me know if there is any other information that you may need.

 Thank you in advance for your help

 Raj

solr autosuggest to stop/filter suggesting the phrases that ends with stopwords

[Rajesh Hazari]

Hi Folks,

Solr Version 4.7+

Do we have any analyzer or filter or any plugin in solr to stop suggesting
the phrase that ends with stopwords?

For ex: If the suggestion are as below for query
http://localhost.com/solr/suggest?q=jazz+a

suggestion: [
jazz and,
jazz at,
jazz at lincoln,
jazz at lincoln center,
jazz artists,
jazz and classic
]

Is there any config or solution to remove only *jazz at* and *jazz and*
phrases so that the final suggestion response looks more sensible!

suggestion: [
jazz at lincoln,
jazz at lincoln center,
jazz artists,
jazz and classic
]

Google does this intelligently :)

I have tested with StopFilterFactory and SuggestStopFilter both of which
filters all of stop terms in the phrases now matter where they appear.

Do i have to come up with a custom plugin or some kind of phrase filter to
do this in solr?

I am on the way to design SuggestStopPhraseFilter and its factory , as we
have existing SuggestStopFilter, and use this in my schema

or do we have any existing plugin or feature that i can use of leverage
from?
*Thanks,*
*Rajesh.*