wt=xml not defaulting the results to xml format

[yaswanth kumar]

Can someone help me on this ASAP? I am using solr 8.2.0 and below is the
snippet from solrconfig.xml for one of the configset, where I am trying to
default the results into xml format but its giving me as a json result.




  all
  10
  
 pkey
 xml


Can some one let me know if I need to do something more to always get a
solr /select query results as XML??
-- 
Thanks & Regards,
Yaswanth Kumar Konathala.
yaswanth...@gmail.com

Re: Cybersecurity Incident Report

[Man with No Name]

You’re absolutely right. Some of these are shadow jars and sone directly
used. Like netty, we're securing the communication using tls and the netty
cve applies.

So going back to the initial question, what would be the best way to
report this, so that it can be looked at?

On Fri, Jul 24, 2020 at 7:35 PM Shawn Heisey  wrote:

> On 7/24/2020 2:35 PM, Man with No Name wrote:
> > This version of jackson is pulled in as a shadow jar. Also solr is using
> > io.netty version 4.1.29.Final which has critical vulnerabilities which
> > are fixed in 4.1.44.
>
> It looks like that shaded jackson library is included in the jar for
> htrace.  I looked through the commit history and learned that htrace is
> included for the HDFS support in Solr.  Which means that if you are not
> using the HDFS capability, then htrace will not be used, so the older
> jackson library will not be used either.
>
> If you are not using TLS connections from SolrCloud to ZooKeeper, then
> your install of Solr will not be using the netty library, and
> vulnerabilities in netty will not apply.
>
> The older version of Guava is pulled in with a jar from carrot2.  If
> your Solr install does not use carrot2 clustering, then that version of
> Guava will never be called.
>
> The commons-compress and tika libraries are only used if you have
> configured the extraction contrib, also known as SolrCell.  This contrib
> module is used to index rich-text documents, such as PDF and Word.
> Because it makes Solr unstable, we strongly recommend that nobody should
> use SolrCell in production.  When rich-text documents need to be
> indexed, it should be accomplished by using Tika outside of Solr... and
> if that recommendation is followed, you can control the version used so
> that the well-known vulnerabilities will not be present.
>
> We have always recommended that Solr should be located in a network
> place that can only be reached by systems and people who are authorized.
>   If that is done, then nobody will be able to exploit any
> vulnerabilities that might exist in Solr unless they first successfully
> break into an authorized system.
>
> We do take these reports of vulnerabilities seriously and close them as
> quickly as we can.
>
> Thanks,
> Shawn
>
-- 
Sent from Gmail for IPhone

Re: org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

[sanjay dutt]

FieldType defined with class solr.SpatialRecursivePrefixTreeFieldType

In this we are adding points only although collection has few fields with 
points data and then other fieldTypes as well.
And one of the queries looks like 
(my_field: [45,-94 TO 46,-93]+OR+my_field: [42,-94 TO 43,-93])

Thanks and Regards,Sanjay Dutt 

On Thursday, August 6, 2020, 12:10:04 AM GMT+5:30, David Smiley 
 wrote:  
 
 What is the Solr field type definition for this field?  And what sort of
spatial data do you add here -- just points or what?

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Aug 3, 2020 at 10:09 PM sanjay dutt
 wrote:

> Hello Solr community,
> On our Production SolrCloud Server, OutOfMemory has been occurring on lot
> of instances. When I download the HEAP DUMP and analyzed it. I got to know
> that in multiple HEAP DUMPS there are lots of instances
> of org.apache.lucene.codecs.blocktree.BlockTreeTermsReader  which has the
> highest retained heap memory and further I have checked the
> outgoing-reference for those objects,
> the  org.apache.lucene.util.fst.FST is the one which occupy 90% of the heap
> memory.
> it's like
> Production HEAP memory :- 12GBout of
> which  org.apache.lucene.codecs.blocktree.BlockTreeTermsReader total retained
> heap :- 7-8 GB(vary from instance to
> instance)and org.apache.lucene.util.fst.FST total retained heap :- 6-7 GB
> Upon further looking I have calculated the total retained heap for
> FieldReader.fieldInfo.name="my_field" is around 7GB. Now this is the same
> reader which also contains reference to org.apache.lucene.util.fst.FST.
> Now "my_field" is the field on which we are performing spatial searches.
> Is spatial searches use FST internally and hence we are seeing lot of heap
> memory used by FST.l only.
> IS there any way we can optimize the spatial searches so that it take less
> memory.
> Can someone please give me any pointer that from where Should I start
> looking to debug the above issue.
> Thanks and Regards,Sanjay Dutt
> Sent from Yahoo Mail on Android
  

Re: Solr 8.5.2 - Solr shards param does not work without localhost

[Shawn Heisey]

On 8/6/2020 6:03 PM, gnandre wrote:

Please ignore the space between. I have updated the calls by removing space
below:

http://my.domain.com/solr/core/select?q=*:*=0=10=
my.domain.com/solr/another_core=*

http://my.domain.com/solr/core/select?q=*:*=0=10=
localhost:8983/solr/another_core=*


Assuming that these are the actual URLs (copy/paste) and not something 
you've typed up as an example...  one of them has port 8983 and the 
other has no port, which would mean it's using port 80.


That looks like it could be a problem.  It takes special effort to get 
Solr listening on port 80.


Thanks,
Shawn

Re: Solr 8.5.2 - Solr shards param does not work without localhost

[gnandre]

Please ignore the space between. I have updated the calls by removing space
below:

http://my.domain.com/solr/core/select?q=*:*=0=10=
my.domain.com/solr/another_core=*

http://my.domain.com/solr/core/select?q=*:*=0=10=
localhost:8983/solr/another_core=*


On Thu, Aug 6, 2020 at 7:59 PM gnandre  wrote:

> Hi,
>
> In Solr 6.3 I was able to use following shards query:
>
> http://my.domain.com/solr/core/select?q=*:*=0=10=
> my.domain.com /solr/another_core=*
>
> Ir does not work in Solr 8.5.2 anymore unless I pass localhost instead of
> my domain in shards param value as follows:
> http://my.domain.com/solr/core/select?q=*:*=0=10=
> localhost:8983  /solr/another_core=*
>
> This is a master-slave setup and not a cloud setup.
>

Re: Solr 8.5.2 - Solr shards param does not work without localhost

[Shawn Heisey]

On 8/6/2020 5:59 PM, gnandre wrote:

http://my.domain.com/solr/core/select?q=*:*=0=10=
my.domain.com /solr/another_core=*

Ir does not work in Solr 8.5.2 anymore unless I pass localhost instead of
my domain in shards param value as follows:
http://my.domain.com/solr/core/select?q=*:*=0=10=
localhost:8983  /solr/another_core=*

This is a master-slave setup and not a cloud setup.


I've set up sharded indexes without SolrCloud before, and I've never 
used "localhost".  Always used FQDN.


When you try it using the name, what shows up in your solr logfile? I 
would assume you're getting some kind of error.  Can you share it?  It 
is likely to be many lines long.


Thanks,
Shawn

Solr 8.5.2 - Solr shards param does not work without localhost

[gnandre]

Hi,

In Solr 6.3 I was able to use following shards query:

http://my.domain.com/solr/core/select?q=*:*=0=10=
my.domain.com /solr/another_core=*

Ir does not work in Solr 8.5.2 anymore unless I pass localhost instead of
my domain in shards param value as follows:
http://my.domain.com/solr/core/select?q=*:*=0=10=
localhost:8983  /solr/another_core=*

This is a master-slave setup and not a cloud setup.

copyField from empty multivalue

[matthew sporleder]

I have a copyField:
 
 

But sometimes preview () is not populated.

It appears that the "catchall" field does not get created when preview
has no content in it.  Can I use required=false or similar on a
copyField?

Thanks,
Matt

Re: Replication of Solr Model and feature store

[Monica Skidmore]

I would be interested in the answer here, as well.  We're using LTR 
successfully on Solr 7.3 and Solr 8.3 in cloud mode, but we're struggling to 
load a simple, test model on 8.3 in master/slave mode.   The FeatureStore 
appears to load, but we're not sure it's loading correctly, either. Here are 
some details from the engineer on our team who is leading that effort:

"I'm getting a ClassCastException when uploading a Model. Using the debugger, 
was able to see the line throwing the exception is: 
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:488)

Apparently it cannot find: org.apache.solr.ltr.model.LinearModel, although the 
features appear to be created without issues with the following class: 
org.apache.solr.ltr.feature.FieldValueFeature

Another thing we were able to see is that the List features has a list 
of null elements, so that made us think there may be some issues when creating 
the instances of Feature.

We had begun to believe this might be related to the fact that we are running 
Solr in Master/Slave config. Was LTR ever tested on non-cloud deployments??
 
Any help is appreciated."

Monica D Skidmore
Lead Engineer, Core Search



CareerBuilder.com  | Blog 
 | Press Room 

 
 
 

On 7/24/20, 7:58 AM, "Christine Poerschke (BLOOMBERG/ LONDON)" 
 wrote:

Hi Krishan,

Could you share what version of Solr you are using?

And I wonder if the observed behaviour could be reproduced e.g. with the 
techproducts example, changes not applying after reload [1] sounds like a bug 
if so.

Hope that helps.

Regards,

Christine

[1] 
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F8_6%2Flearning-to-rank.html%23applying-changesdata=01%7C01%7CMonica.Skidmore%40careerbuilder.com%7C65581e5e79414c90832508d82fc8ce21%7C7cc1677566a34e8b80fd5b1f1db15061%7C0sdata=mMqgPhnkjb8h7ETQNaySOBJQ8x%2FP2dtzM%2FgSE1K1FZg%3Dreserved=0

From: solr-user@lucene.apache.org At: 07/22/20 14:00:59To:  
solr-user@lucene.apache.org
Subject: Re: Replication of Solr Model and feature store

Adding more details here

I need some help on how to enable the solr LTR model and features on all
nodes of a solr cluster.

I am unable to replicate the model and the feature store though from any
master to its slaves with the replication API ? And unable to find any
documentation for the same. Is replication possible?

Without replication, would I have to individually update all nodes of a
cluster ? Or can the feature and model files be read as a resource (like
config or schema) so that I can replicate the file or add the file to my
deployments.


On Wed, Jul 22, 2020 at 5:53 PM krishan goyal  wrote:

> Bump. Any one has an idea how to proceed here ?
>
> On Wed, Jul 8, 2020 at 5:41 PM krishan goyal 
> wrote:
>
>> Hi,
>>
>> How do I enable replication of the model and feature store ?
>>
>> Thanks
>> Krishan
>>
>

Backups in SolrCloud using snapshots of individual cores?

[Bram Van Dam]

Hey folks,

Been reading up about the various ways of creating backups. The whole
"shared filesystem for Solrcloud backups"-thing is kind of a no-go in
our environment, so I've been looking for ways around that, and here's
what I've come up with so far:

1. Stop applications from writing to solr

2. Commit everything

3. Identify a single core for each shard in each collection

4. Snapshot that core using CREATESNAPSHOT in the Collections API

5. Once complete, re-enable application write access to Solr

6. Create a backup from these snapshots using the replication handler's
backup function (replication?command=backup=mySnapshot)

7. Put the backups somewhere safe

8. Clean up snapshots


This seems ... too good to be true? I've seen so many threads about how
hard it is to create backups in SolrCloud on this mailing list over the
years, but this seems pretty straightforward? Am I missing some
glaringly obvious reason why this will fail catastrophically?

Using Solr 7.7 in this case.

Feedback much appreciated!

Thanks,

 - Bram

Re: Pull Replica compaints about UpdateLog being disabled when DocBasedVersionConstraintsProcessorFactory

[Erick Erickson]

See: SOLR-14715

> On Aug 6, 2020, at 1:58 AM, Tomás Fernández Löbbe  
> wrote:
> 
> This is an interesting bug. I’m wondering if we can completely skip the
> initialization of UpdateRequestProcessorFactories in PULL replicas...
> 
> On Wed, Aug 5, 2020 at 8:40 AM Erick Erickson 
> wrote:
> 
>> Offhand, this looks like a bug, please raise a JIRA.
>> 
>> You said: " We also have DocBasedVersionConstraintsProcessorFactory in our
>> UpdateProcessorChain for optimistic Concurrency.”
>> 
>> Optimistic concurrency is automatically enofrced on the _version_ field.
>> The intent of this processor factory is to allow you finer control over
>> optimistic concurrency by explicitly defining/populating fields. I do
>> wonder whether you need this factory at all. If the intent is that any
>> document with the same  is updated with optimistic concurrency,
>> you don’t need it at all.
>> 
>> Best,
>> Erick
>> 
>>> On Aug 4, 2020, at 2:17 PM, harjags
>>  wrote:
>>> 
>>> DocBasedVersionConstraintsProcessorFactory
>> 
>> 

Suggester not suggesting but spellchecker is

[Philip Smith]

Hello,

Any advice on the following suggester not suggesting issue would be very
welcome.

I can get suggestions when using a spell checker but not when using the
suggester. Both types are querying the same suggestion field.

I'm using the following settings:

 



  


  



  


I've tried using different field types for suggestion. I read that the
field shouldn't be heavily processed with stemmers for instance, I tried
with string but it made no difference.

  
  
  
  


I've tried with about 6 different examples from online, and none return
results, below is an example of one, the other examples were variations
using FuzzyLookupFactory instead.

  
  
  mySuggester
  FSTLookupFactory
  DocumentDictionaryFactory
  suggestion
  string

  


  

  true
  10
 mySuggester


  suggest5

  

http://localhost:8983/solr/events/suggest5?suggest.dictionary=mySuggester=true=true=edu
http://localhost:8983/solr/events/suggest5?q=edu

both return

{
  "responseHeader":{
"status":0,
"QTime":4},
  "suggest":{"mySuggester":{
  "edu":{
"numFound":0,
"suggestions":[]


spell checker returns a couple of results for this.
I was restarting solr after making any changes.

This is the setup for the spellchecker:

  
  string
  
  suggest
  org.apache.solr.spelling.suggest.Suggester
 FuzzyLookupFactory
 string
  suggestion
  0.1
  spellchecker
  freq
  true
  0.5
  
  
 

  
  
  on
  suggest
  true
  true
  10
  true
  10
  5
  
  
  suggest
  
  


I'm trying to get an old version 4 config working updated to work with 8.5
Am I missing out by not being able to use the dedicated suggester?
Is the field type for suggestion optimal?

Many thanks in advance.

Best,
Phil.













  
  string
  
  suggest
  org.apache.solr.spelling.suggest.Suggester
  FuzzyLookupFactory
  string
  suggestion
  0.1
  spellchecker
  freq
  true
  0.5