Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Yasufumi Mizoguchi]

Hi, Shawn,
(Sorry, I have sent this your private email address...)

Thanks for your reply.

I understood what you are saying. However, at least, I think it strange 
that UnifiedSolrHighlighter
returns the same error when choosing ", " as the field delimiter in 
hl.fl (e.g. hl.fl=name,%20manu).
This is because UnifiedSolrHighlighter detects that there is a 
zero-length string between "," and " ",

and treats the string as a field name.
Is this a correct behavior?

Thanks,
Yasufumi


On 2017/09/05 12:21 AM, Shawn Heisey wrote:

On 9/3/2017 10:31 PM, Yasufumi Mizoguchi wrote:

I am testing UnifiedHighlighter(hl.method=unified) with Solr 6.6 and
found that the highlighter returns following error when hl.fl
parameter has undefined fields.
The error occurs even if hl.fl parameter has ", "( + )
as a field delimiter. (e.g. hl.fl=name, manu)
Is this a bug? I think that the highlighter should set OffsetSource as
"NONE_NEEDED", if fields are not available

I think Solr is doing exactly what it should in this instance.  If you
ask Solr to use a field that it doesn't have, immediately returning an
error message is the right thing to do.  If Solr were to ignore the
error and proceed with the request anyway, then the person making the
request might never know that they have made a mistake.

Thanks,
Shawn

Re: Error opening new searcher due to LockObtainFailedException

[Erick Erickson]

Hmmm. oddly another poster was seeing this due to permissions issues,
although I don't know why that would clear up after a while. But it's
something to check.

Erick

On Wed, Aug 30, 2017 at 3:24 PM, Sundeep T  wrote:
> Hello,
>
> Occasionally we are seeing errors opening new searcher for certain solr
> cores. Whenever this happens, we are unable to query or ingest new data
> into these cores. It seems to clear up after some time though. The root
> cause seems to be - *"org.apache.lucene.store.LockObtainFailedException:
> Lock held by this virtual machine:
> /opt/solr/volumes/data9/7d50b38e114af075-core-24/data/index/write.lock"*
>
> Below is the full stack trace. Any ideas on what could be going on that
> causes such an exception and how to mitigate this? thanks a lot for your
> help!
>
> Unable to create core
> [7d50b38e114af075-core-24],trace=org.apache.solr.common.SolrException:
> Unable to create core [7d50b38e114af075-core-24]
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:903)
> at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1167)
> at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:296)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.(SolrCore.java:952)
> at org.apache.solr.core.SolrCore.(SolrCore.java:816)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:890)
> ... 30 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1891)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2011)
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1041)
> at org.apache.solr.core.SolrCore.(SolrCore.java:925)
> ... 32 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock held by
> this virtual machine:
> /opt/solr/volumes/data9/7d50b38e114af075-core-24/data/index/write.lock
> at
> org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:127)
> at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
> at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
> at
> org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
> at org.apache.lucene.index.IndexWriter.(IndexWriter.java:804)
> at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:125)
> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:100)
> at
> 

Re: write.lock file appears and solr wont open

[Erick Erickson]

Gah, thanks for letting us know. I can't tell you how often
permissions issues have tripped me up. You're right, it does seem like
there could be a better error message though.

Erick

On Mon, Sep 4, 2017 at 1:55 PM, Phil Scadden  wrote:
> We finally got a resolution to this - trivial but related to trying to do 
> things by remote control. The solr process did not have the permissions to 
> write to the core that was imported. When it tried to create the lock file it 
> failed. The Solr code obviously assumes that file create failure means file 
> already exists rather than perhaps insufficient permissions. Checking for 
> file existence would result in a more informative message but I am guessing 
> the test/production setup when developers are not allowed access to the 
> servers is reasonably unique (I hope so anyway because it sucks).
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: Saturday, 26 August 2017 9:15 a.m.
> To: solr-user 
> Subject: Re: write.lock file appears and solr wont open
>
> Odd. The way core discovery works, it starts at SOLR_HOME and recursively 
> descends the directories. Whenever the recursion finds a "core.properties" 
> file it says "Aha, this must be a core". From there it assumes the data 
> directory is immediately below where it found the core.properties file in the 
> absence of any dataDir overrides.
>
> So how the write.lock file is getting preserved across Solr restarts is a 
> mystery to me. Doing a "kill -9" is one way to make that happen if it is done 
> at just the wrong time, but that's unlikely in what you're describing.
>
> Are you totally sure that there were no old Solr processes running?
> And there have been some issues in the past where the log display of the 
> admin UI hold on to errors and displays them after the problem has been 
> fixed. I'm assuming you can't query the new core, is that correct? Because if 
> you can query the core then _something_ has the index open. I'm grasping at 
> straws here mind you.
>
> Best,
> Erick
>
> On Thu, Aug 24, 2017 at 9:02 PM, Phil Scadden  wrote:
>> SOLR_HOME is /var/www/solr/data
>> The zip was actually the entire data directory which also included 
>> configsets. And yes core.properties is in var/www/solr/data/prindex (just 
>> has single line name=prindex, in it). No other cores are present.
>> The data directory should have been unzipped before the solr instance was 
>> started (I cant actually touch the machine so communicating via a deployment 
>> document but the operator usually follows every step to the letter.
>> The sequence was:
>> mkdir /var/www/solr
>> sudo bash ./install_solr_service.sh solr-6.5.1.tgz -i /opt/local -d
>> /var/www/solr edit /etc/default/solr.in.sh to set various items. (esp
>> SOLR_HOME and to set SOLR_PID_DIR to /var/www/solr) unzip the data
>> directory service solr start.
>>
>> No other instance of solr installed.
>>
>> Notice: This email and any attachments are confidential and may not be used, 
>> published or redistributed without the prior written consent of the 
>> Institute of Geological and Nuclear Sciences Limited (GNS Science). If 
>> received in error please destroy and immediately notify GNS Science. Do not 
>> copy or disclose the contents.
> Notice: This email and any attachments are confidential and may not be used, 
> published or redistributed without the prior written consent of the Institute 
> of Geological and Nuclear Sciences Limited (GNS Science). If received in 
> error please destroy and immediately notify GNS Science. Do not copy or 
> disclose the contents.

RE: write.lock file appears and solr wont open

[Phil Scadden]

We finally got a resolution to this - trivial but related to trying to do 
things by remote control. The solr process did not have the permissions to 
write to the core that was imported. When it tried to create the lock file it 
failed. The Solr code obviously assumes that file create failure means file 
already exists rather than perhaps insufficient permissions. Checking for file 
existence would result in a more informative message but I am guessing the 
test/production setup when developers are not allowed access to the servers is 
reasonably unique (I hope so anyway because it sucks).

-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Saturday, 26 August 2017 9:15 a.m.
To: solr-user 
Subject: Re: write.lock file appears and solr wont open

Odd. The way core discovery works, it starts at SOLR_HOME and recursively 
descends the directories. Whenever the recursion finds a "core.properties" file 
it says "Aha, this must be a core". From there it assumes the data directory is 
immediately below where it found the core.properties file in the absence of any 
dataDir overrides.

So how the write.lock file is getting preserved across Solr restarts is a 
mystery to me. Doing a "kill -9" is one way to make that happen if it is done 
at just the wrong time, but that's unlikely in what you're describing.

Are you totally sure that there were no old Solr processes running?
And there have been some issues in the past where the log display of the admin 
UI hold on to errors and displays them after the problem has been fixed. I'm 
assuming you can't query the new core, is that correct? Because if you can 
query the core then _something_ has the index open. I'm grasping at straws here 
mind you.

Best,
Erick

On Thu, Aug 24, 2017 at 9:02 PM, Phil Scadden  wrote:
> SOLR_HOME is /var/www/solr/data
> The zip was actually the entire data directory which also included 
> configsets. And yes core.properties is in var/www/solr/data/prindex (just has 
> single line name=prindex, in it). No other cores are present.
> The data directory should have been unzipped before the solr instance was 
> started (I cant actually touch the machine so communicating via a deployment 
> document but the operator usually follows every step to the letter.
> The sequence was:
> mkdir /var/www/solr
> sudo bash ./install_solr_service.sh solr-6.5.1.tgz -i /opt/local -d
> /var/www/solr edit /etc/default/solr.in.sh to set various items. (esp
> SOLR_HOME and to set SOLR_PID_DIR to /var/www/solr) unzip the data
> directory service solr start.
>
> No other instance of solr installed.
>
> Notice: This email and any attachments are confidential and may not be used, 
> published or redistributed without the prior written consent of the Institute 
> of Geological and Nuclear Sciences Limited (GNS Science). If received in 
> error please destroy and immediately notify GNS Science. Do not copy or 
> disclose the contents.
Notice: This email and any attachments are confidential and may not be used, 
published or redistributed without the prior written consent of the Institute 
of Geological and Nuclear Sciences Limited (GNS Science). If received in error 
please destroy and immediately notify GNS Science. Do not copy or disclose the 
contents.

Re: edismax-with-bq-complexphrase-query not working in solrconfig.xml search handler

[Doug Turnbull]

The problem is that the ${q} macro syntax is interpreted by Solr as a Java
Property. Thus these two syntaxes conflict when we encode the macro
substitution into the solrconfig.xml. See
https://cwiki.apache.org/confluence/display/solr/Configuring+solrconfig.xml

The way to escape seems to be to indicate ${q} is the default value to use,
ie:

  {!complexphrase df=title}"${q:${q}}"

I must say as a user new to this syntax, this is a surprising complexity.
Also, while a great feature, macro substitution doesn't seem particularly
well documented and i took us a while to figure this out. Should a syntax
that doesn't conflict with Java properties be considered? Perhaps sunset
the original syntax? At the least document how to port your macros to
solrconfig?

Best!
-Doug


On Mon, Sep 4, 2017 at 1:16 PM Bertrand Rigaldies <
brigald...@opensourceconnections.com> wrote:

> Hi there, on Solr 6.6.0, using the built-in "techproducts" example:
>
> bin/solr start -f -e techproducts
>
> I can successfully search with URL-based bq as shown in the URL (bq setting
> in *bold*) below:
>
> http://localhost:8983/solr/techproducts/select?
> *bq={!complexphrase%20inOrder=true%20df=name}%22${q}%22*
> =on=edismax=*,score=on=iPod%20Mini=json
>
> where I see the expected ComplexPhraseQuery expression in the parsed query:
>
> "parsedquery":"(+(DisjunctionMaxQuery((text:ipod))
> DisjunctionMaxQuery((text:mini))) *ComplexPhraseQuery(\"iPod Mini\")*
> )/no_coord"
>
> However, *I have been unable to accomplish the same query using a search
> handler *in solrconfig.xml, as shown below:
>
>   
> 
>   ALL
>   10
>
>   
>   edismax
>
>   
>   name
>
>   
>   name, score
>
>   
>   
>   *{!complexphrase inOrder=true df=name}"${q}"*
>
>   
>   
> 
>   
>
> The expression {!complexphrase inOrder=true
> df=name}"${q}" errors out when loading solrconfig.xml with the
> following error:
>
> Error: org.apache.solr.common.SolrException: No system property or default
> value specified for q value:{!complexphrase inOrder=true df=name}"${q}"
>
> as if Solr is looking for a Java property "q"!?
>
> The variation shows below without the macro expansion, but assuming a
> normal localParam substitution, seems to be taking "$q" as a literate:
>
> Setting:
>   {!complexphrase inOrder=true}name:"$q"
> Produces:
>   "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
> DisjunctionMaxQuery((name:mini))) *ComplexPhraseQuery(\"$q\")*)/no_coord"
>
> The variation with an attempt to escape the macro expansion does not work
> either:
>
> Setting:
>   {!complexphrase inOrder=true
> df=name}"\\$\\{q\\}"
> Produces:
>   "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
> DisjunctionMaxQuery((name:mini)))* ComplexPhraseQuery(\"\\$\\{q\\}\"))*
> /no_coord"
>
> Also, the following variations do *not* produce the expected
> ComplexPhraseQuery statement in the parsed query:
> Settings:
>   {!complexphrase inOrder=true}name:$q
>   {!complexphrase inOrder=true df=name}$q
>   {!complexphrase inOrder=true}name:($q)
>   {!complexphrase inOrder=true df=name}($q)
>   {!complexphrase inOrder=true}name:\"$q\"
>   {!complexphrase inOrder=true df=name}\"$q\"
>   {!complexphrase inOrder=true df=name v="$q"}
>   {!complexphrase inOrder=true df=name v=\"$q\"}
> Produces:
>   "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
> DisjunctionMaxQuery((name:mini))) *name:q*)/no_coord"
>
> And finally, also not producing the expected ComplexPhraseQuery statement:
> Setting:
>   {!complexphrase inOrder=true df=name v=$q}
> Produces:
>   "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
> DisjunctionMaxQuery((name:mini))) *(name:ipod name:mini)*)/no_coord"
>
> The documentation for ComplexPhraseQuery
>  >
> stipulates
> some required "escaping": Special care has to be given when escaping:
> clauses between double quotes (usually whole query) is parsed twice, these
> parts have to be escaped as twice. eg "foo\\: bar\\^"
>
> hence, it is possible that I have not used the proper escaping syntax.
>
> It is troubling that I cannot use the same URL parameter expression in a
> search handler to accomplish the same effect, a strong assumption of mine
> in how Solr can be used.
>
> Any suggestion, comment, or similar experience? Does it look like a bug?
>
> Thank you,
> Bertrand
>
-- 
Consultant, OpenSource Connections. Contact info at
http://o19s.com/about-us/doug-turnbull/; Free/Busy (http://bit.ly/dougs_cal)

edismax-with-bq-complexphrase-query not working in solrconfig.xml search handler

[Bertrand Rigaldies]

Hi there, on Solr 6.6.0, using the built-in "techproducts" example:

bin/solr start -f -e techproducts

I can successfully search with URL-based bq as shown in the URL (bq setting
in *bold*) below:

http://localhost:8983/solr/techproducts/select?
*bq={!complexphrase%20inOrder=true%20df=name}%22${q}%22*
=on=edismax=*,score=on=iPod%20Mini=json

where I see the expected ComplexPhraseQuery expression in the parsed query:

"parsedquery":"(+(DisjunctionMaxQuery((text:ipod))
DisjunctionMaxQuery((text:mini))) *ComplexPhraseQuery(\"iPod Mini\")*
)/no_coord"

However, *I have been unable to accomplish the same query using a search
handler *in solrconfig.xml, as shown below:

  

  ALL
  10

  
  edismax

  
  name

  
  name, score

  
  
  *{!complexphrase inOrder=true df=name}"${q}"*

  
  

  

The expression {!complexphrase inOrder=true
df=name}"${q}" errors out when loading solrconfig.xml with the
following error:

Error: org.apache.solr.common.SolrException: No system property or default
value specified for q value:{!complexphrase inOrder=true df=name}"${q}"

as if Solr is looking for a Java property "q"!?

The variation shows below without the macro expansion, but assuming a
normal localParam substitution, seems to be taking "$q" as a literate:

Setting:
  {!complexphrase inOrder=true}name:"$q"
Produces:
  "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
DisjunctionMaxQuery((name:mini))) *ComplexPhraseQuery(\"$q\")*)/no_coord"

The variation with an attempt to escape the macro expansion does not work
either:

Setting:
  {!complexphrase inOrder=true df=name}"\\$\\{q\\}"
Produces:
  "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
DisjunctionMaxQuery((name:mini)))* ComplexPhraseQuery(\"\\$\\{q\\}\"))*
/no_coord"

Also, the following variations do *not* produce the expected
ComplexPhraseQuery statement in the parsed query:
Settings:
  {!complexphrase inOrder=true}name:$q
  {!complexphrase inOrder=true df=name}$q
  {!complexphrase inOrder=true}name:($q)
  {!complexphrase inOrder=true df=name}($q)
  {!complexphrase inOrder=true}name:\"$q\"
  {!complexphrase inOrder=true df=name}\"$q\"
  {!complexphrase inOrder=true df=name v="$q"}
  {!complexphrase inOrder=true df=name v=\"$q\"}
Produces:
  "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
DisjunctionMaxQuery((name:mini))) *name:q*)/no_coord"

And finally, also not producing the expected ComplexPhraseQuery statement:
Setting:
  {!complexphrase inOrder=true df=name v=$q}
Produces:
  "parsedquery":"(+(DisjunctionMaxQuery((name:ipod))
DisjunctionMaxQuery((name:mini))) *(name:ipod name:mini)*)/no_coord"

The documentation for ComplexPhraseQuery

stipulates
some required "escaping": Special care has to be given when escaping:
clauses between double quotes (usually whole query) is parsed twice, these
parts have to be escaped as twice. eg "foo\\: bar\\^"

hence, it is possible that I have not used the proper escaping syntax.

It is troubling that I cannot use the same URL parameter expression in a
search handler to accomplish the same effect, a strong assumption of mine
in how Solr can be used.

Any suggestion, comment, or similar experience? Does it look like a bug?

Thank you,
Bertrand

Re: unordered autocomplete search

[Walter Underwood]

This should probably be a feature of the analyzing infix suggester. 

Right now, the fuzzy suggester is broken with the file dictionary, so we can’t 
use fuzzy suggestions at all.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


> On Sep 4, 2017, at 4:50 AM, Mikhail Khludnev  wrote:
> 
> You probably can override AnalyzingInfixSuggester.finishQuery(Builder,
> boolean) where you can rip clauses from builder and change TermQueries/
> PrefixQuery to FuzzyQueries. Let me know how it works, please.
> 
> On Mon, Sep 4, 2017 at 2:27 PM, Niraj Aswani  wrote:
> 
>> Hi Mikhali,
>> 
>> Thank you very much for your quick response.
>> 
>> This does seem to fix the issue of unordered words in the autocomplete,
>> however, then I loose the capability of matching misspelled words.
>> 
>> Is there anything that can be done to combine the functionality of the two?
>> 
>> Regards,
>> Niraj
>> 
>> 
>> 
>> On Mon, Sep 4, 2017 at 11:13 AM, Mikhail Khludnev  wrote:
>> 
>>> The first question is, "sourceLocation">test.dict makes sense?
>>> Then, for me, it looks like a case for AnalyzingInfixLookupFactory
>>> https://lucidworks.com/2015/03/04/solr-suggester/ rather than
>>> 
>>>   - The FuzzyLookupFactory that creates suggestions for misspelled words
>>>   in fields.
>>> 
>>> 
>>> 
>>> 
>>> On Mon, Sep 4, 2017 at 11:57 AM, Niraj Aswani 
>>> wrote:
>>> 
 Hi,
 
 I am using solr 5.5.4.
 
 I would like to perform an unordered autocomplete search however it
>> seems
 that it only suggests phrases that start with my term in the query.
 
 For example, in my dictionary I have,
 
 *lamp desk*
 
 When searching for the term "lamp", it is able to show the the
>> suggestion
 "lamp desk" as the suggestion is starting with the word "lamp".
>> However,
 when I search for the term "desk", it doesn't show any suggestion.
 
 My field in the *managed-schema* is as below:
 
 >>> positionIncrementGap="100">
  


  
  

  
 
 
 My controller in *solconfig.xml* is defined as the following:
 
 
 
  
default
FuzzyLookupFactory
test.dict
suggest_field
  
 
 
 >> startup="lazy">
  
true
10
  
  
suggest
  
 
 
 The *query* I fire is:
 
 http://localhost:8983/solr/core1/suggest?q=desk=true
 
 is there anyway, I can get the above scenario working?
 
 Many thanks,
 Niraj
 
>>> 
>>> 
>>> 
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>> 
>> 
> 
> 
> 
> -- 
> Sincerely yours
> Mikhail Khludnev

Re: Performance Test

[Walter Underwood]

That is what I do. Use production logs. I have a JMeter script that sets a 
constant request rate. 

Before each load benchmark, I reload the collection to clear the caches, then 
run 2000 warming queries from the logs. After that, I start the benchmark.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


> On Sep 4, 2017, at 7:18 AM, Dave  wrote:
> 
> Get the raw logs from normal use, script out something to replicate the 
> searches and have it fork to as many cores as the solr server has is what I'd 
> do. 
> 
> 
> 
>> On Sep 4, 2017, at 5:26 AM, Daniel Ortega  
>> wrote:
>> 
>> I would recommend you Solrmeter cloud
>> 
>> This fork supports solr cloud:
>> https://github.com/idealista/solrmeter/blob/master/README.md
>> 
>> Disclaimer: This fork was developed by idealista, the company where I work
>> 
>> El El lun, 4 sept 2017 a las 11:18, Selvam Raman 
>> escribió:
>> 
>>> Hi All,
>>> 
>>> which is the best tool for solr perfomance test. I want to identify how
>>> much load my solr could handle and how many concurrent users can query on
>>> solr.
>>> 
>>> Please suggest.
>>> 
>>> --
>>> Selvam Raman
>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>> 

Re: How to add domain name like "ocean-solr-cd.dit.ocean.slb.com" in solr URL

[Shawn Heisey]

On 9/4/2017 4:35 AM, LAD, SAGAR wrote:
> How to add domain name like "ocean-solr-cd.dit.ocean.slb.com" in solr
> URL. Could you please let me know steps to add domain name?

The name by which you access Solr is not within the control of Solr.

SolrCloud is the only possible exception.  Each node in a cloud
registers itself in the zookeeper database with a host name, a TCP port,
and a server context.  If SolrCloud is left to do things without
configuration, it will register an IP address.  If you want it to
register a name instead, edit solr.xml to change/add the "host"
definition in the solrcloud section.

https://wiki.apache.org/solr/Solr.xml%204.4%20and%20beyond

Note that if you are editing an existing cloud to change the host names,
you're probably going to have to manually edit the zookeeper database to
remove the definitions under the old names.

Your Solr server should *not* be reachable from the open Internet if you
can avoid it.

Thanks,
Shawn

Re: query with wild card with AND taking lot of time

[Shawn Heisey]

On 9/1/2017 5:24 PM, Walter Underwood wrote:
> Hmm. Solr really should convert an fq of “a AND b” to separate “a” and “b” fq 
> filters. That should be a simple special-case rewrite. It might take less 
> time to implement than explaining it to everyone.
>
> Well, I guess then we’d have to explain how it wasn’t really necessary to 
> send separate fq params…

I don't think it's a good idea for Solr to attempt optimizations like
this for everything.  Sure, there are plenty of times that people ask
Solr to do something, not realizing that what they have asked for is a
bad idea, but what about the person who *does* know exactly what they
have asked Solr to do, and actually did intend to do it that way? 
Turning one filter query into five filter queries does sound like a good
idea, but those filters will all run in parallel, which depending on the
use case might overwhelm CPU resources.

This is probably going to sound insane, but I like the fact that Solr
gives me plenty of rope with which to hang myself.  It means I have full
access to all the power of the system.  I know that Solr will do exactly
what I tell it to do, even if I don't actually understand all the
instructions I've given it.

Thanks,
Shawn

Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Shawn Heisey]

On 9/3/2017 10:31 PM, Yasufumi Mizoguchi wrote:
> I am testing UnifiedHighlighter(hl.method=unified) with Solr 6.6 and
> found that the highlighter returns following error when hl.fl
> parameter has undefined fields.
> The error occurs even if hl.fl parameter has ", "( + )
> as a field delimiter. (e.g. hl.fl=name, manu)
> Is this a bug? I think that the highlighter should set OffsetSource as
> "NONE_NEEDED", if fields are not available 

I think Solr is doing exactly what it should in this instance.  If you
ask Solr to use a field that it doesn't have, immediately returning an
error message is the right thing to do.  If Solr were to ignore the
error and proceed with the request anyway, then the person making the
request might never know that they have made a mistake.

Thanks,
Shawn

Re: slow solr facet processing

[Yonik Seeley]

On Mon, Sep 4, 2017 at 6:38 AM, Toke Eskildsen  wrote:
> On Mon, 2017-09-04 at 13:21 +0300, Ere Maijala wrote:
>> Thanks for the insight, Yonik. I can confirm that #2 is true. I ran
>>
>> 
>>
>> and after it completed I was able to retrieve 2000 values in 17ms.
>
> Very interesting. Is this on spinning disks or SSD? Is your index data
> cached in memory? What I am aiming at is if this is primarily a "many
> relatively slow random access"-thing or more due to the way DocValues
> are represented in the segments (the codec).

It's due to this (see comments in UnInvertedField):
*   To further save memory, the terms (the actual string values) are
not all stored in
*   memory, but a TermIndex is used to convert term numbers to term values only
*   for the terms needed after faceting has completed.  Only every
128th term value
*   is stored, along with its corresponding term number, and this is used as an
*   index to find the closest term and iterate until the desired number is hit

There's probably a number of ways we can speed this up somewhat:
- optimize how much memory is used to store the term index and use the
savings to store more than every 128th term
- store the terms contiguously in block(s)
- don't store the whole term, only store what's needed to seek to the
Nth term correctly
- when retrieving many terms, sort them first and convert from ord->str in order

-Yonik

Managed Resources separated by core when using configset

[Alessandro Hoss]

Hello,

I've a situation where I need the same configuration (schema.xml and
solrconfig.xml) for all the cores in my solr instance, and I can manage
this with the same configset.
BUT, in my case the synonyms and stopwords should be managed by tenant/core
(each tenant is a solr core), and can be different for each one.

I was trying to use managed resources, but I guess it doesn't handle the
core when using configset.
Once I pass the core name in CRUD operations, it looks like a bug, because
it's not handling different resources, and if I don't reload the core right
before a PUT request, for example, the next PUT in other core will override
the previous and ignore them. Here's what I'm trying:

Create core1 using techproducts configset:
curl "
http://localhost:8983/solr/admin/cores?action=CREATE=core1=core1=sample_techproducts_configs
"

Create core2 using techproducts configset:
curl "
http://localhost:8983/solr/admin/cores?action=CREATE=core2=core2=sample_techproducts_configs
"

Check the current stopwords (they are the same at this point): --this
requests will be used again later
curl "http://localhost:8983/solr/core1/schema/analysis/stopwords/english;
curl "http://localhost:8983/solr/core2/schema/analysis/stopwords/english;

Add a stopword in core1:
curl -X PUT -H 'Content-type:application/json' --data-binary '["zz1"]' "
http://localhost:8983/solr/core1/schema/analysis/stopwords/english;

Check the current stopwords (now the core2 does NOT have the inserted
stopword - and shouldn't, because the request was specifically for core1)

ADD a stopword in core2:
curl -X PUT -H 'Content-type:application/json' --data-binary '["zz2"]' "
http://localhost:8983/solr/core2/schema/analysis/stopwords/english;

Check the current stopwords (now both cores have what I expected - "zz1" in
core1 and "zz2" in core2)

The problem is, now when I reload the core1, it will override the stopwords
with the last updated version, which means it'll lose the "zz1" word.
curl "http://localhost:8983/solr/admin/cores?action=RELOAD=core1;
curl "http://localhost:8983/solr/core1/schema/analysis/stopwords/english;

Did I make myself clear? :)
Can someone confirm if this is a bug or if I doing something wrong?

Thanks in advance,
Alessandro Hoss

Re: Performance Test

[Dave]

Get the raw logs from normal use, script out something to replicate the 
searches and have it fork to as many cores as the solr server has is what I'd 
do. 



> On Sep 4, 2017, at 5:26 AM, Daniel Ortega  wrote:
> 
> I would recommend you Solrmeter cloud
> 
> This fork supports solr cloud:
> https://github.com/idealista/solrmeter/blob/master/README.md
> 
> Disclaimer: This fork was developed by idealista, the company where I work
> 
> El El lun, 4 sept 2017 a las 11:18, Selvam Raman 
> escribió:
> 
>> Hi All,
>> 
>> which is the best tool for solr perfomance test. I want to identify how
>> much load my solr could handle and how many concurrent users can query on
>> solr.
>> 
>> Please suggest.
>> 
>> --
>> Selvam Raman
>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>> 

Re: slow solr facet processing

[Ere Maijala]

Toke Eskildsen kirjoitti 4.9.2017 klo 13.38:

On Mon, 2017-09-04 at 13:21 +0300, Ere Maijala wrote:

Thanks for the insight, Yonik. I can confirm that #2 is true. I ran



and after it completed I was able to retrieve 2000 values in 17ms.


Very interesting. Is this on spinning disks or SSD? Is your index data
cached in memory? What I am aiming at is if this is primarily a "many
relatively slow random access"-thing or more due to the way DocValues
are represented in the segments (the codec).


I indexed a few million new/changed records, and the performance is back 
to slow. Upside is that I can test again with a slow server.


It's spinning disks on a SAN, and the full index doesn't fit into 
memory. I don't see any IO wait, and repeated attempts are just as slow 
even though I would have thought the relevant parts would be cached in 
memory. During testing and reporting the results I've always discarded 
the very first requests since they're always slower than subsequent 
repeats due to there being another test index on the same server. Maybe 
worth noting is that while there's no IO wait, there is fairly high CPU 
usage for Solr's Java process hovering around 100% if I repeat the 
request in a loop.


I took a quick sample with VisualVM, and the top hotspots are:

org.apache.solr.search.facet.UnInvertedField.getCounts()	32.079956	7,356 
ms (32.1%)	7,356 ms	7,655 ms	7,655 ms
org.apache.lucene.util.PriorityQueue.downHeap()	30.232546	6,932 ms 
(30.2%)	6,932 ms	6,932 ms	6,932 ms
org.apache.lucene.index.MultiTermsEnum.pushTop()	11.628195	2,666 ms 
(11.6%)	2,666 ms	11,177 ms	11,177 ms
org.apache.lucene.index.MultiTermsEnum$TermMergeQueue.fillTop()	9.079571 
2,082 ms (9.1%)	2,082 ms	2,082 ms	2,082 ms
org.apache.lucene.store.ByteBufferGuard.getBytes()	4.176216	957 ms 
(4.2%)	957 ms	957 ms	957 ms
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.next() 
2.6867974	616 ms (2.7%)	616 ms	616 ms	616 ms
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.scanToTermLeaf() 
1.393562	319 ms (1.4%)	319 ms	319 ms	319 ms
org.apache.lucene.util.fst.ByteSequenceOutputs.read()	1.2111844	277 ms 
(1.2%)	277 ms	277 ms	277 ms


(sorry if that looks bad in the email)

I'm building another index on a higher-end server that can load the full 
index to memory and will retest with that. But note that this index has 
docValues disabled as facet.method=uif seems to only cause trouble if 
docValues are enabled.


--Ere

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

Re: unordered autocomplete search

[Mikhail Khludnev]

You probably can override AnalyzingInfixSuggester.finishQuery(Builder,
boolean) where you can rip clauses from builder and change TermQueries/
PrefixQuery to FuzzyQueries. Let me know how it works, please.

On Mon, Sep 4, 2017 at 2:27 PM, Niraj Aswani  wrote:

> Hi Mikhali,
>
> Thank you very much for your quick response.
>
> This does seem to fix the issue of unordered words in the autocomplete,
> however, then I loose the capability of matching misspelled words.
>
> Is there anything that can be done to combine the functionality of the two?
>
> Regards,
> Niraj
>
>
>
> On Mon, Sep 4, 2017 at 11:13 AM, Mikhail Khludnev  wrote:
>
> > The first question is, "sourceLocation">test.dict makes sense?
> > Then, for me, it looks like a case for AnalyzingInfixLookupFactory
> > https://lucidworks.com/2015/03/04/solr-suggester/ rather than
> >
> >- The FuzzyLookupFactory that creates suggestions for misspelled words
> >in fields.
> >
> >
> >
> >
> > On Mon, Sep 4, 2017 at 11:57 AM, Niraj Aswani 
> > wrote:
> >
> > > Hi,
> > >
> > > I am using solr 5.5.4.
> > >
> > > I would like to perform an unordered autocomplete search however it
> seems
> > > that it only suggests phrases that start with my term in the query.
> > >
> > > For example, in my dictionary I have,
> > >
> > > *lamp desk*
> > >
> > > When searching for the term "lamp", it is able to show the the
> suggestion
> > > "lamp desk" as the suggestion is starting with the word "lamp".
> However,
> > > when I search for the term "desk", it doesn't show any suggestion.
> > >
> > > My field in the *managed-schema* is as below:
> > >
> > >  > > positionIncrementGap="100">
> > >   
> > > 
> > > 
> > >   
> > >   
> > > 
> > >   
> > > 
> > >
> > > My controller in *solconfig.xml* is defined as the following:
> > >
> > > 
> > > 
> > >   
> > > default
> > > FuzzyLookupFactory
> > > test.dict
> > > suggest_field
> > >   
> > > 
> > >
> > >  > startup="lazy">
> > >   
> > > true
> > > 10
> > >   
> > >   
> > > suggest
> > >   
> > > 
> > >
> > > The *query* I fire is:
> > >
> > > http://localhost:8983/solr/core1/suggest?q=desk=true
> > >
> > > is there anyway, I can get the above scenario working?
> > >
> > > Many thanks,
> > > Niraj
> > >
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>



-- 
Sincerely yours
Mikhail Khludnev

Re: unordered autocomplete search

[Niraj Aswani]

Hi Mikhali,

Thank you very much for your quick response.

This does seem to fix the issue of unordered words in the autocomplete,
however, then I loose the capability of matching misspelled words.

Is there anything that can be done to combine the functionality of the two?

Regards,
Niraj



On Mon, Sep 4, 2017 at 11:13 AM, Mikhail Khludnev  wrote:

> The first question is, "sourceLocation">test.dict makes sense?
> Then, for me, it looks like a case for AnalyzingInfixLookupFactory
> https://lucidworks.com/2015/03/04/solr-suggester/ rather than
>
>- The FuzzyLookupFactory that creates suggestions for misspelled words
>in fields.
>
>
>
>
> On Mon, Sep 4, 2017 at 11:57 AM, Niraj Aswani 
> wrote:
>
> > Hi,
> >
> > I am using solr 5.5.4.
> >
> > I would like to perform an unordered autocomplete search however it seems
> > that it only suggests phrases that start with my term in the query.
> >
> > For example, in my dictionary I have,
> >
> > *lamp desk*
> >
> > When searching for the term "lamp", it is able to show the the suggestion
> > "lamp desk" as the suggestion is starting with the word "lamp". However,
> > when I search for the term "desk", it doesn't show any suggestion.
> >
> > My field in the *managed-schema* is as below:
> >
> >  > positionIncrementGap="100">
> >   
> > 
> > 
> >   
> >   
> > 
> >   
> > 
> >
> > My controller in *solconfig.xml* is defined as the following:
> >
> > 
> > 
> >   
> > default
> > FuzzyLookupFactory
> > test.dict
> > suggest_field
> >   
> > 
> >
> >  startup="lazy">
> >   
> > true
> > 10
> >   
> >   
> > suggest
> >   
> > 
> >
> > The *query* I fire is:
> >
> > http://localhost:8983/solr/core1/suggest?q=desk=true
> >
> > is there anyway, I can get the above scenario working?
> >
> > Many thanks,
> > Niraj
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: slow solr facet processing

[Toke Eskildsen]

On Mon, 2017-09-04 at 13:21 +0300, Ere Maijala wrote:
> Thanks for the insight, Yonik. I can confirm that #2 is true. I ran
> 
> 
> 
> and after it completed I was able to retrieve 2000 values in 17ms.

Very interesting. Is this on spinning disks or SSD? Is your index data
cached in memory? What I am aiming at is if this is primarily a "many
relatively slow random access"-thing or more due to the way DocValues
are represented in the segments (the codec).

- Toke Eskildsen, Royal Danish Library

RE: Solr uses lots of shared memory!

[Markus Jelsma]

Hello Kevin, Rick,

These are interesting points indeed. But this thread is about shared memory, 
not virtual memory.

Any value higher than 0 for MALLOC_ARENA_MAX only reduces virtual memory 
consumption, from 22 GB to 16 GB. There are no differences in shared memory nor 
resident memory.

Although intrestesting, it unfortunately does not answer the question.

To answer Rick's question, the difference with MALLOC_ARENA_MAX=2 is less 
virtual memory but no changes in queries/second.

Thanks,
Markus

-Original message-
> From:Rick Leir 
> Sent: Sunday 3rd September 2017 17:08
> To: solr-user@lucene.apache.org
> Subject: Re: Solr uses lots of shared memory!
> 
> Hi all
> Malloc has a lock while it is active in the heap. If there is more than one 
> thread, and malloc finds the lock in use, then it avoids waiting on the lock 
> by creating a new 'arena' to hold its heap. My understanding is that a 
> process with multiple threads which are all active users of malloc will 
> eventually have an arena per thread. If you limit the number of arenas, you 
> may suffer delays waiting on locks. 
> 
> But this needs performance testing. My experience was with C++, not with a 
> JVM. I would be interested to know if setting MALLOC_ARENA_MAX=2 makes a 
> difference to performance.
> Cheers -- Rick
> 
> On September 2, 2017 1:15:38 PM EDT, Kevin Risden  
> wrote:
> >I haven't looked at reproducing this locally, but since it seems like
> >there haven't been any new ideas decided to share this in case it
> >helps:
> >
> >I noticed in Travis CI [1] they are adding the environment variable
> >MALLOC_ARENA_MAX=2 and so I googled what that configuration did. To my
> >surprise, I came across a stackoverflow post [2] about how glibc could
> >actually be the case and report memory differently. I then found a
> >Hadoop issue HADOOP-7154 [3] about setting this as well to reduce
> >virtual memory usage. I found some more cases where this has helped as
> >well [4], [5], and [6]
> >
> >[1]
> >https://docs.travis-ci.com/user/build-environment-updates/2017-09-06/#Added
> >[2]
> >https://stackoverflow.com/questions/10575342/what-would-cause-a-java-process-to-greatly-exceed-the-xmx-or-xss-limit
> >[3]
> >https://issues.apache.org/jira/browse/HADOOP-7154?focusedCommentId=14505792
> >[4] https://github.com/cloudfoundry/java-buildpack/issues/320
> >[5] https://devcenter.heroku.com/articles/tuning-glibc-memory-behavior
> >[6]
> >https://www.ibm.com/developerworks/community/blogs/kevgrig/entry/linux_glibc_2_10_rhel_6_malloc_may_show_excessive_virtual_memory_usage?lang=en
> >Kevin Risden
> >
> >
> >On Thu, Aug 24, 2017 at 10:19 AM, Markus Jelsma
> > wrote:
> >> Hello Bernd,
> >>
> >> According to the man page, i should get a list of stuff in shared
> >memory if i invoke it with just a PID. Which shows a list of libraries
> >that together account for about 25 MB's shared memory usage. Accoring
> >to ps and top, the JVM uses 2800 MB shared memory (not virtual), that
> >leaves 2775 MB unaccounted for. Any ideas? Anyone else to reproduce it
> >on a freshly restarted node?
> >>
> >> Thanks,
> >> Markus
> >>
> >>
> >>   PID USER  PR  NI    VIRT    RES    SHR S  %CPU %MEM TIME+
> >COMMAND
> >> 18901 markus    20   0 14,778g 4,965g 2,987g S 891,1 31,7  20:21.63
> >java
> >>
> >> 0x55b9a17f1000  6K 
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
> >> 0x7fdf1d314000  182K   
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libsunec.so
> >> 0x7fdf1e548000  38K    
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libmanagement.so
> >> 0x7fdf1e78e000  94K    
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libnet.so
> >> 0x7fdf1e9a6000  75K    
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libnio.so
> >> 0x7fdf5cd6e000  34K    
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libzip.so
> >> 0x7fdf5cf77000  46K    
> >/lib/x86_64-linux-gnu/libnss_files-2.24.so
> >> 0x7fdf5d189000  46K    
> >/lib/x86_64-linux-gnu/libnss_nis-2.24.so
> >> 0x7fdf5d395000  90K /lib/x86_64-linux-gnu/libnsl-2.24.so
> >> 0x7fdf5d5ae000  34K    
> >/lib/x86_64-linux-gnu/libnss_compat-2.24.so
> >> 0x7fdf5d7b7000  187K   
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so
> >> 0x7fdf5d9e6000  70K    
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libverify.so
> >> 0x7fdf5dbf8000  30K /lib/x86_64-linux-gnu/librt-2.24.so
> >> 0x7fdf5de0  90K /lib/x86_64-linux-gnu/libgcc_s.so.1
> >> 0x7fdf5e017000  1063K   /lib/x86_64-linux-gnu/libm-2.24.so
> >> 0x7fdf5e32  1553K  
> >/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
> >> 0x7fdf5e6a8000  15936K 
> >/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> >> 0x7fdf5f5ed000  139K   
> >/lib/x86_64-linux-gnu/libpthread-2.24.so
> >> 

How to add domain name like "ocean-solr-cd.dit.ocean.slb.com" in solr URL

[LAD, SAGAR]

Hi Team,
How to add domain name like "ocean-solr-cd.dit.ocean.slb.com" in solr URL. 
Could you please let me know steps to add domain name?

Regards
_
[Email_CBE.gif]Sagar Lad | Capgemini Technology Services India Limited | Pune
Schlumberger, Sogeti India
Main: +91 20 2760 1000 extn 2012331 | Cell: +91 8983724089 +91 98 199 64738
www.capgemini.com
People matter, results count.
___
[50years]



Connect with Capgemini:
[Picto_Blog][Picto_Twitter][Picto_Facebook][Picto_LinkedIn][Picto_Slideshare][Picto_YouTube]















This message contains information that may be privileged or confidential and is 
the property of the Capgemini Group. It is intended only for the person to whom 
it is addressed. If you are not the intended recipient, you are not authorized 
to read, print, retain, copy, disseminate, distribute, or use this message or 
any part thereof. If you receive this message in error, please notify the 
sender immediately and delete all copies of this message.

Re: slow solr facet processing

[Ere Maijala]

Yonik Seeley kirjoitti 1.9.2017 klo 17.03:> On Fri, Sep 1, 2017 at 9:17 
AM, Ere Maijala  wrote:

>> I spoke a bit too soon. Now I see why I didn't see any improvement from
>> facet.method=uif before: its performance seems to depend heavily on 
how many
>> facets are returned. With an index of 6 million records and the 
facet having

>> 1960 buckets:
>>
>> facet.limit=20 takes 4ms
>> facet.limit=200 takes ~100ms
>> facet.limit=2000 takes ~1300ms
>>
>> So, for some uses it provides a nice boost, but if you need to fetch 
more

>> than a few top items, it doesn't perform properly.
>
> Another thought on this one:
> If it does slow down more than 4.x when requesting many items, it's 
either

> 1) a bug introduced at some point
> 2) not actually slower, but due to the 6.6 index having more segments
> (ord->string conversion needs to merge multiple term enumerators, so
> more segments == slower)
>
> If you could check #2, that would be great!  If it doesn't seem to be
> the problem, could you open up a new JIRA issue for this?
>
Thanks for the insight, Yonik. I can confirm that #2 is true. I ran



and after it completed I was able to retrieve 2000 values in 17ms.

Does this mean we should have a very aggressive merge policy? That's 
something I haven't tweaked, and it's not quite clear to me what would 
be the best way to achieve consistently low number of segments.


I encountered one issue with some further testing. I assume this is a 
bug: Trying to use facet.method=uif with a solr.DateRangeField causes 
the following exception:


2017-09-04 12:50:33.246 ERROR (qtp1205044462-18602) [   x:biblio2] 
o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: 
Exception during facet.field: search_daterange_mv
at 
org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:809)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.request.SimpleFacets$3.execute(SimpleFacets.java:742)
at 
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:818)
at 
org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:326)
at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:274)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:304)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

at org.eclipse.jetty.server.Server.handle(Server.java:534)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at 

Re: unordered autocomplete search

[Mikhail Khludnev]

The first question is, "sourceLocation">test.dict makes sense?
Then, for me, it looks like a case for AnalyzingInfixLookupFactory
https://lucidworks.com/2015/03/04/solr-suggester/ rather than

   - The FuzzyLookupFactory that creates suggestions for misspelled words
   in fields.




On Mon, Sep 4, 2017 at 11:57 AM, Niraj Aswani  wrote:

> Hi,
>
> I am using solr 5.5.4.
>
> I would like to perform an unordered autocomplete search however it seems
> that it only suggests phrases that start with my term in the query.
>
> For example, in my dictionary I have,
>
> *lamp desk*
>
> When searching for the term "lamp", it is able to show the the suggestion
> "lamp desk" as the suggestion is starting with the word "lamp". However,
> when I search for the term "desk", it doesn't show any suggestion.
>
> My field in the *managed-schema* is as below:
>
>  positionIncrementGap="100">
>   
> 
> 
>   
>   
> 
>   
> 
>
> My controller in *solconfig.xml* is defined as the following:
>
> 
> 
>   
> default
> FuzzyLookupFactory
> test.dict
> suggest_field
>   
> 
>
> 
>   
> true
> 10
>   
>   
> suggest
>   
> 
>
> The *query* I fire is:
>
> http://localhost:8983/solr/core1/suggest?q=desk=true
>
> is there anyway, I can get the above scenario working?
>
> Many thanks,
> Niraj
>



-- 
Sincerely yours
Mikhail Khludnev

Re: Performance Test

[Daniel Ortega]

I would recommend you Solrmeter cloud

This fork supports solr cloud:
https://github.com/idealista/solrmeter/blob/master/README.md

Disclaimer: This fork was developed by idealista, the company where I work

El El lun, 4 sept 2017 a las 11:18, Selvam Raman 
escribió:

> Hi All,
>
> which is the best tool for solr perfomance test. I want to identify how
> much load my solr could handle and how many concurrent users can query on
> solr.
>
> Please suggest.
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Performance Test

[Selvam Raman]

Hi All,

which is the best tool for solr perfomance test. I want to identify how
much load my solr could handle and how many concurrent users can query on
solr.

Please suggest.

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

unordered autocomplete search

[Niraj Aswani]

Hi,

I am using solr 5.5.4.

I would like to perform an unordered autocomplete search however it seems
that it only suggests phrases that start with my term in the query.

For example, in my dictionary I have,

*lamp desk*

When searching for the term "lamp", it is able to show the the suggestion
"lamp desk" as the suggestion is starting with the word "lamp". However,
when I search for the term "desk", it doesn't show any suggestion.

My field in the *managed-schema* is as below:


  


  
  

  


My controller in *solconfig.xml* is defined as the following:



  
default
FuzzyLookupFactory
test.dict
suggest_field
  



  
true
10
  
  
suggest
  


The *query* I fire is:

http://localhost:8983/solr/core1/suggest?q=desk=true

is there anyway, I can get the above scenario working?

Many thanks,
Niraj