ExtractingRequestHandler AND NLP

[Kyle W. Bolin]

I am trying to implement the NLP functionality within the Solr 
ExtractingRequestHandler  and  the Tika framework
I am using PDF documents to index and have been successful in extracting and 
indexing the content but have not been successful in engaging the NLP routines. 
  I have reached the point where I even trying to generate an exception just to 
validate my understanding of the interfaces.
I have included  parts of my solrconfig and tika.config . Also,  I am using the 
techproducts example and Solr 6.3.0


solrconfig.xml
--- NLP Models  en-ner-organization.bin etc

  

  
  

solrconfig.xml

  

  true
  attr_
  tika-config.xml
  
  true
  links
  ignored_

  


Tika-Config.xml





text/plain
text/html
application/xhtml+xml
application/pdf

does velocity search ui work ?

[win harrington]

https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI
I'm using Solr 6.2,  bin/solr -e techproducts
This doesn't work for me."When using the sample_techproducts_configs  config 
set, you can access the Velocity sample Search UI"
 http://localhost:8983/solr/techproducts/browse
 

Re: does velocity search ui work ?

[win harrington]

Thank you. I set all the techproductsfiles to permissions 777 and it 
startedworking.   :) 

On Wednesday, November 16, 2016 3:18 PM, Erick Erickson 
 wrote:
 

 Works fine for me. What exactly do you see? And what does your solr
log show when you try to access the browse handler?

Best,
Erick

On Wed, Nov 16, 2016 at 11:35 AM, win harrington
 wrote:
> https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI
> I'm using Solr 6.2,  bin/solr -e techproducts
> This doesn't work for me."When using the sample_techproducts_configs  config 
> set, you can access the Velocity sample Search UI"
>  http://localhost:8983/solr/techproducts/browse
>


   

Hardware size in solrcloud

[Mugeesh Husain]

I have lots of document i dont know now how much it would be in future. for
the inilial stage, I am looking for hardware details(assumption).

I are looking forward to setting up a billion document(1 billion approx)
solr index and the size is 7GB.

Can you please suggest the hardware details as per experience.
1. OS(32/64bit):
2. Processor:
3. RAM:
4. No of physical servers/systems :


Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Hardware-size-in-solrcloud-tp4306169.html
Sent from the Solr - User mailing list archive at Nabble.com.

velocity is working now

[win harrington]

I couldn't get Velocity Search to 
work.https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI
I set all the files to permissions 777 andVelocity works now.  :)
I'm just experimenting now. I'll look laterto see what the problem was.

index dir of core xxx is already locked.

[Chetas Joshi]

Hi,

I have a SolrCloud (on HDFS) of 52 nodes. I have 3 collections each with 50
shards and maxShards per node for every collection is 1.

I am having problem restarting a solr shard for a collection.

When I restart, there is always a particular shard of a particular
collection that remains down. The 2 shards on the same host for the rest of
the collections are up and running.

Before restarting, I delete all the write.lock files from the data dir. But
every time I restart I get the same exception.

index dir yyy of core xxx is already locked. The most likely cause is
another Solr server (or another solr core in this server) also configured
to use this directory; other possible causes may be specific to lockType:
hdfs

Thanks!

Re: does velocity search ui work ?

[Erick Erickson]

Works fine for me. What exactly do you see? And what does your solr
log show when you try to access the browse handler?

Best,
Erick

On Wed, Nov 16, 2016 at 11:35 AM, win harrington
 wrote:
> https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI
> I'm using Solr 6.2,  bin/solr -e techproducts
> This doesn't work for me."When using the sample_techproducts_configs  config 
> set, you can access the Velocity sample Search UI"
>  http://localhost:8983/solr/techproducts/browse
>

Re: Basic Auth for Solr Streaming Expressions

[sandeep mukherjee]

[SOLR-9779] Basic auth in not supported in Streaming Expressions - ASF JIRA
  
|  
|   
|   
|   ||

   |

  |
|  
|   |  
[SOLR-9779] Basic auth in not supported in Streaming Expressions - ASF JIRA
   |   |

  |

  |

 

I have created the above jira ticket for the base auth support in solr 
streaming expressions.
ThanksSandeep 

On Wednesday, November 16, 2016 8:22 AM, sandeep mukherjee 
 wrote:
 

  blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px 
#715FFA solid !important; padding-left:1ex !important; background-color:white 
!important; }  Nope never got past the login screen. Will create one today.


Sent from Yahoo Mail for iPhone


On Wednesday, November 16, 2016, 8:17 AM, Kevin Risden 
 wrote:

Was a JIRA ever created for this? I couldn't find it searching.

One that is semi related is SOLR-8213 for SolrJ JDBC auth.

Kevin Risden

On Wed, Nov 9, 2016 at 8:25 PM, Joel Bernstein  wrote:

> Thanks for digging into this, let's create a jira ticket for this.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Nov 9, 2016 at 6:23 PM, sandeep mukherjee <
> wiredcit...@yahoo.com.invalid> wrote:
>
> > I have more progress since my last mail. I figured out that  in the
> > StreamContext object there is a way to set the SolrClientCache object
> which
> > keep reference to all the CloudSolrClient where I can set a reference to
> > HttpClient which sets the Basic Auth header. However the problem is,
> inside
> > the SolrClientCache there is no way to set your own version of
> > CloudSolrClient with BasicAuth enabled. Unfortunately, SolrClientCache
> has
> > no set method which takes a CloudSolrClient object.
> > So long story short we need an API in SolrClientCache to
> > accept CloudSolrClient object from user.
> > Please let me know if there is a better way to enable Basic Auth when
> > using StreamFactory as mentioned in my previous email.
> > Thanks much,Sandeep
> >
> >    On Wednesday, November 9, 2016 11:44 AM, sandeep mukherjee
> >  wrote:
> >
> >
> >  Hello everyone,
> > I trying to find the documentation for Basic Auth plugin for Solr
> > Streaming expressions. But I'm not able to find it in the documentation
> > anywhere. Could you please point me in right direction of how to enable
> > Basic auth for Solr Streams?
> > I'm creating StreamFactory as follows: I wonder how and where can I
> > specify Basic Auth username and password
> > @Bean
> > public StreamFactory streamFactory() {
> >    SolrConfig solrConfig = ConfigManager.getNamedConfig("solr",
> > SolrConfig.class);
> >
> >    return new StreamFactory().withDefaultZkHost(solrConfig.
> > getConnectString())
> >            .withFunctionName("gatherNodes", GatherNodesStream.class);
> > }
> >
> >
> >
>

 


   

Re: Hardware size in solrcloud

[Kevin Risden]

First question: is your initial sizing correct?

7GB/1 billion = 7 bytes per document? That would be basically 7 characters?

Anyway there are lots of variables regarding sizing. The typical response
is:

https://lucidworks.com/blog/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/

Kevin Risden

On Wed, Nov 16, 2016 at 1:12 PM, Mugeesh Husain  wrote:

> I have lots of document i dont know now how much it would be in future. for
> the inilial stage, I am looking for hardware details(assumption).
>
> I are looking forward to setting up a billion document(1 billion approx)
> solr index and the size is 7GB.
>
> Can you please suggest the hardware details as per experience.
> 1. OS(32/64bit):
> 2. Processor:
> 3. RAM:
> 4. No of physical servers/systems :
>
>
> Thanks
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Hardware-size-in-solrcloud-tp4306169.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: index dir of core xxx is already locked.

[Erick Erickson]

bq: Before restarting, I delete all the write.lock files from the data dir. But
every time I restart I get the same exception.

First, this shouldn't be necessary. Are you by any chance killing the
Solr instances with
the equivalent of "kill -9"? Allow them to shut down gracefully. That
said, until recently
the bin/solr script would kill them forcefully after 5 seconds which
is too short an interval.

But the error really is telling you that somehow two or more Solr
cores are pointing at the
same data directory. Whichever one gets there first will block any
later cores with the
message you see. So check your core.properties files and your HDFS magic to see
how this is occurring would be my first guess.

Best,
Erick

On Wed, Nov 16, 2016 at 1:38 PM, Chetas Joshi  wrote:
> Hi,
>
> I have a SolrCloud (on HDFS) of 52 nodes. I have 3 collections each with 50
> shards and maxShards per node for every collection is 1.
>
> I am having problem restarting a solr shard for a collection.
>
> When I restart, there is always a particular shard of a particular
> collection that remains down. The 2 shards on the same host for the rest of
> the collections are up and running.
>
> Before restarting, I delete all the write.lock files from the data dir. But
> every time I restart I get the same exception.
>
> index dir yyy of core xxx is already locked. The most likely cause is
> another Solr server (or another solr core in this server) also configured
> to use this directory; other possible causes may be specific to lockType:
> hdfs
>
> Thanks!

Re: Detecting schema errors while adding documents

[Erick Erickson]

You're probably thinking about SOLR-445 (yes, 445. This issue has
been hanging around for that long)

But that's for much more recent versions than 4.10.3 so without looking
at the Solr logs I don't think there's much of a way to tell.

The awkward way is, when a batch fails re-submit the docs either in
some kind of binary pattern or one at a time until you find the offending
doc.

Best,
Erick

On Wed, Nov 16, 2016 at 10:02 AM, Mike Thomsen  wrote:
> We're stuck on Solr 4.10.3 (Cloudera bundle). Is there any way to detect
> with SolrJ when a document added to the index violated the schema? All we
> see when we look at the stacktrace for the SolrException that comes back is
> that it contains messages about an IOException when talking to the solr
> nodes. Solr is up and running, and the documents are only invalid because I
> added a Java statement to make a field invalid for testing purposes. When I
> remove that statement, the indexing happens just fine.
>
> Any way to do this? I seem to recall that at least in newer versions of
> Solr it would tell you more about the specific error.
>
> Thanks,
>
> Mike

Re: index dir of core xxx is already locked.

[Chetas Joshi]

I don't kill the solr instance forcefully using "kill -9".

I checked the core.properties file for that shard. The content is different
from the core.properties file for all the other shards.
It has the following two lines which are different

config=solrconfig.xml

schema=schema.xml

In other shards, it is

collection.configName=v4 (name I have given to the config)

name=collectionName_shardNumber_replica1

Should I modify this file before restarting the Cloud?

There is a strange thing I just observed about the data dir of the shard
that is not coming up. There is an addition index dir that has been created

hdfs://Ingest/solr53/collection/core_node32/data/index/index/

The size and content is same as of

hdfs://Ingest/solr53/collection/core_node32/data/index/


What could be the reason of this extra dir? Should I delete it?


Thanks!


On Wed, Nov 16, 2016 at 1:51 PM, Erick Erickson 
wrote:

> bq: Before restarting, I delete all the write.lock files from the data
> dir. But
> every time I restart I get the same exception.
>
> First, this shouldn't be necessary. Are you by any chance killing the
> Solr instances with
> the equivalent of "kill -9"? Allow them to shut down gracefully. That
> said, until recently
> the bin/solr script would kill them forcefully after 5 seconds which
> is too short an interval.
>
> But the error really is telling you that somehow two or more Solr
> cores are pointing at the
> same data directory. Whichever one gets there first will block any
> later cores with the
> message you see. So check your core.properties files and your HDFS magic
> to see
> how this is occurring would be my first guess.
>
> Best,
> Erick
>
> On Wed, Nov 16, 2016 at 1:38 PM, Chetas Joshi 
> wrote:
> > Hi,
> >
> > I have a SolrCloud (on HDFS) of 52 nodes. I have 3 collections each with
> 50
> > shards and maxShards per node for every collection is 1.
> >
> > I am having problem restarting a solr shard for a collection.
> >
> > When I restart, there is always a particular shard of a particular
> > collection that remains down. The 2 shards on the same host for the rest
> of
> > the collections are up and running.
> >
> > Before restarting, I delete all the write.lock files from the data dir.
> But
> > every time I restart I get the same exception.
> >
> > index dir yyy of core xxx is already locked. The most likely cause is
> > another Solr server (or another solr core in this server) also configured
> > to use this directory; other possible causes may be specific to lockType:
> > hdfs
> >
> > Thanks!
>

Re: index dir of core xxx is already locked.

[Erick Erickson]

You really need to go through your Solr logs for the
shard(s) in question very carefully. There'll be a lot of
information dumped out, including paths used for
everything.

I suspect you've unknowingly created this situation when
trying to set up Solr, HDFS or whatever but I can't really say
what it would be without inspecting your entire installation,
which I can;t do remotely.

Best,
Erick

On Wed, Nov 16, 2016 at 3:01 PM, Chetas Joshi  wrote:
> I don't kill the solr instance forcefully using "kill -9".
>
> I checked the core.properties file for that shard. The content is different
> from the core.properties file for all the other shards.
> It has the following two lines which are different
>
> config=solrconfig.xml
>
> schema=schema.xml
>
> In other shards, it is
>
> collection.configName=v4 (name I have given to the config)
>
> name=collectionName_shardNumber_replica1
>
> Should I modify this file before restarting the Cloud?
>
> There is a strange thing I just observed about the data dir of the shard
> that is not coming up. There is an addition index dir that has been created
>
> hdfs://Ingest/solr53/collection/core_node32/data/index/index/
>
> The size and content is same as of
>
> hdfs://Ingest/solr53/collection/core_node32/data/index/
>
>
> What could be the reason of this extra dir? Should I delete it?
>
>
> Thanks!
>
>
> On Wed, Nov 16, 2016 at 1:51 PM, Erick Erickson 
> wrote:
>
>> bq: Before restarting, I delete all the write.lock files from the data
>> dir. But
>> every time I restart I get the same exception.
>>
>> First, this shouldn't be necessary. Are you by any chance killing the
>> Solr instances with
>> the equivalent of "kill -9"? Allow them to shut down gracefully. That
>> said, until recently
>> the bin/solr script would kill them forcefully after 5 seconds which
>> is too short an interval.
>>
>> But the error really is telling you that somehow two or more Solr
>> cores are pointing at the
>> same data directory. Whichever one gets there first will block any
>> later cores with the
>> message you see. So check your core.properties files and your HDFS magic
>> to see
>> how this is occurring would be my first guess.
>>
>> Best,
>> Erick
>>
>> On Wed, Nov 16, 2016 at 1:38 PM, Chetas Joshi 
>> wrote:
>> > Hi,
>> >
>> > I have a SolrCloud (on HDFS) of 52 nodes. I have 3 collections each with
>> 50
>> > shards and maxShards per node for every collection is 1.
>> >
>> > I am having problem restarting a solr shard for a collection.
>> >
>> > When I restart, there is always a particular shard of a particular
>> > collection that remains down. The 2 shards on the same host for the rest
>> of
>> > the collections are up and running.
>> >
>> > Before restarting, I delete all the write.lock files from the data dir.
>> But
>> > every time I restart I get the same exception.
>> >
>> > index dir yyy of core xxx is already locked. The most likely cause is
>> > another Solr server (or another solr core in this server) also configured
>> > to use this directory; other possible causes may be specific to lockType:
>> > hdfs
>> >
>> > Thanks!
>>

Re: Handling ampersands in searches.

[Erick Erickson]

Why do you think that the porter stemmer is involved here? That
takes tokens and tries to reduce them to their base form through
a set of rules. My guess is that the & just falls outside all rules so
is passed through unimpeded.

This is where the admin/analysis page is invaluable. If you look at
your type you'll notice that you have different options on
WordDelimiterFilterFactory for query and index time, in particular
"preserveOriginal" is 0 at index time and 1 at query.

So you get the tokens
light
fit

in your index and

light
&
fit

at query time.

Then since the query is looking for all three terms it fails. I'm
also guessing you have mm=100% or the default op set to
AND in your edismax configuration.

Anyway, this all kind of starts with choosing WhitespaceTokenizerFactory
as your tokenizer. StandardTokenizerFactory will (I think) remove the
ampersand in both cases. You can also use a CharFilterFactory to apply
some filtering to characters before anything starts going through the
analysis chain (NOTE: this is a CharFilter, not a Filter! See something
like PatternReplaceCharFilterFacotry at
https://cwiki.apache.org/confluence/display/solr/CharFilterFactories#CharFilterFactories-solr.PatternReplaceCharFilterFactory)

Best,
Erick

On Wed, Nov 16, 2016 at 9:34 AM, Callum Lamb  wrote:
> I'm having an issue where searches that contain ampersands aren't being
> handled correctly. I need them to be dropped at index time *AND* query
> time. When documents come in and are indexed the ampersands are
> successfully dropped when they go into my stemmed field (When I facet on
> the stemmed field they aren't in the list), but when I actually search with
> a term containing an ampersand, I get no results.
>
> E.g. I search for the string "light fit" or "light and fit" then I get
> results, but when I search for "light & fit" I get none. Even though the
> SnowballPorterFilterFactory should be dropping it at query time like it
> does for the "and" and all 3 queries *should* be equivalent.
>
> I've tried adding a synonym such that shows in
> my _schema_analysis_synonyms_default.json (I only have one default file) in
> both this form and its inverse as well:
>
> "and":[
>
>   "&",
>   "and"],
>
>
> I've also tried adding the StopWord filter to my fieldtype with & in the
> stopwords (though this shouldn't be necessary because the SnowBallPorter
> should be dropping it anyway) and it still doesn't work.
>
> Is there some kind of special handling I need for ampersands? I'm thinking
> that Solr must be interpreting it as some kind of operator and I need to
> tell Solr that it's actually literal text so the SnowBallPorter knows to
> drop it. Using backslashes or url encoding instead doesn't work though.
> Does anyone have any ideas?
>
> I can obviously just remove any ampersands from the q before I submit the
> query to Solr and get the correct results, so this is not a game breaking
> problem, but i'm more curious to *why* this is happening and how to fix it
> correctly.
>
> Cheers,
>
> Callum.
>
> Extra info:
>
> I'm using Solr 5.5.2 in cloud mode.
>
> The q in the queries is specified like this and are parsed the following
> way:
>
> "rawquerystring":"stemmed_description:light & fit", "querystring":"
> stemmed_description:light & fit", "parsedquery":"(+(+stemmed_description:light
> +DisjunctionMaxQuery((stemmed_description:&)) +DisjunctionMaxQuery((
> stemmed_description:fit/no_coord", "parsedquery_toString":"+(+
> stemmed_description:light +(stemmed_description:&) +(stemmed_description
> :fit))",
>
> I have a stemmed field defined in my schema (schema version 1.5) defined
> like this:
>
>  stored="false" required="false" multiValued="true"/>
>
> with a field type defined like this:
>
> 
>  positionIncrementGap="100" omitNorms="true">
>   
> 
> 
> 
>  catenateWords="1"
> preserveOriginal="0"
> splitOnNumerics="0"/>
> 
>  />
>  language="English"/>
>   
>   
> 
> 
> 
>  catenateWords="1"
> preserveOriginal="1"
> splitOnNumerics="0"/>
> 
>
>  language="English"/>
>   
> 
>
> --
>
> Mintel Group Ltd | 11 Pilgrim Street | London | EC4V 6RN
> Registered in England: Number 1475918. | VAT Number: GB 232 9342 72
>
> Contact details for our other offices can be found at
> http://www.mintel.com/office-locations.
>
> This email and any attachments may include content that is confidential,
> privileged
> or otherwise protected under applicable law. Unauthorised disclosure,
> copying, distribution
> or use of the contents is prohibited and may be unlawful. If you have
> received this email in error,
> including without appropriate authorisation, then please reply to the
> sender about the error
> and delete this email and any attachments.
>

Re: Basic Auth for Solr Streaming Expressions

[Kevin Risden]

Thanks Sandeep!

Kevin Risden

On Wed, Nov 16, 2016 at 3:33 PM, sandeep mukherjee <
wiredcit...@yahoo.com.invalid> wrote:

> [SOLR-9779] Basic auth in not supported in Streaming Expressions - ASF JIRA
>
> |
> |
> |
> |   ||
>
>|
>
>   |
> |
> |   |
> [SOLR-9779] Basic auth in not supported in Streaming Expressions - ASF JIRA
>|   |
>
>   |
>
>   |
>
>
>
> I have created the above jira ticket for the base auth support in solr
> streaming expressions.
> ThanksSandeep
>
> On Wednesday, November 16, 2016 8:22 AM, sandeep mukherjee
>  wrote:
>
>
>   blockquote, div.yahoo_quoted { margin-left: 0 !important;
> border-left:1px #715FFA solid !important; padding-left:1ex !important;
> background-color:white !important; }  Nope never got past the login screen.
> Will create one today.
>
>
> Sent from Yahoo Mail for iPhone
>
>
> On Wednesday, November 16, 2016, 8:17 AM, Kevin Risden <
> compuwizard...@gmail.com> wrote:
>
> Was a JIRA ever created for this? I couldn't find it searching.
>
> One that is semi related is SOLR-8213 for SolrJ JDBC auth.
>
> Kevin Risden
>
> On Wed, Nov 9, 2016 at 8:25 PM, Joel Bernstein  wrote:
>
> > Thanks for digging into this, let's create a jira ticket for this.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Wed, Nov 9, 2016 at 6:23 PM, sandeep mukherjee <
> > wiredcit...@yahoo.com.invalid> wrote:
> >
> > > I have more progress since my last mail. I figured out that  in the
> > > StreamContext object there is a way to set the SolrClientCache object
> > which
> > > keep reference to all the CloudSolrClient where I can set a reference
> to
> > > HttpClient which sets the Basic Auth header. However the problem is,
> > inside
> > > the SolrClientCache there is no way to set your own version of
> > > CloudSolrClient with BasicAuth enabled. Unfortunately, SolrClientCache
> > has
> > > no set method which takes a CloudSolrClient object.
> > > So long story short we need an API in SolrClientCache to
> > > accept CloudSolrClient object from user.
> > > Please let me know if there is a better way to enable Basic Auth when
> > > using StreamFactory as mentioned in my previous email.
> > > Thanks much,Sandeep
> > >
> > >On Wednesday, November 9, 2016 11:44 AM, sandeep mukherjee
> > >  wrote:
> > >
> > >
> > >  Hello everyone,
> > > I trying to find the documentation for Basic Auth plugin for Solr
> > > Streaming expressions. But I'm not able to find it in the documentation
> > > anywhere. Could you please point me in right direction of how to enable
> > > Basic auth for Solr Streams?
> > > I'm creating StreamFactory as follows: I wonder how and where can I
> > > specify Basic Auth username and password
> > > @Bean
> > > public StreamFactory streamFactory() {
> > >SolrConfig solrConfig = ConfigManager.getNamedConfig("solr",
> > > SolrConfig.class);
> > >
> > >return new StreamFactory().withDefaultZkHost(solrConfig.
> > > getConnectString())
> > >.withFunctionName("gatherNodes", GatherNodesStream.class);
> > > }
> > >
> > >
> > >
> >
>
>
>
>
>
>

"add and limit" update modifier or scripted update like elasticsearch

[Dorian Hoxha]

Hi,

Is there an "add and limit" update modifier (couldn't find in docs) ? If
not, can I run a script to update a document (still couldn't find anything)
? If not, how should I do that  (custom plugin? )?

Thank You

How many versions do you stay behind in production for better stability ?

[Dorian Hoxha]

Hi,

I see that there is a new release on every lucene release. Do you always
use the latest version since it may have bugs (ex most cassandra
productions are old compared to latest `stable` version because they're not
stable). How much behind do you usually stay ? (ex: 6.3 just came out, and
you need to be in production after 1 month, will you upgrade on dev if you
don't need any new feature?)

Thank You

Parent child relationship, where children aren't nested but separate (like elasticsearch)

[Dorian Hoxha]

Hi,

I'm not finding a way to support parent-child like es does (using
blockjoin)? I've seen some blogs

with having children as nested inside the parent-document, but I want to
freely crud childs/parents as separate documents (i know that nested also
writes separate documents) and have a special field to link them + manually
route them to the same shard.

Is this possible/available ?

Thank You

Delta indexing with child documents with help of cacheImpl="SortedMapBackedCache"

[Aniket Khare]

Hi,

I am using solr DIH for indexing the Parent-Child relation data and using
cacheImpl="SortedMapBackedCache".
For Full data indexinf I am using command clean="true" and for delta I am
using command full-import and clean="false".
So the same queries are being executed for fulland delta and indexing
working properly.
The issue which we are facing is where for a perticuler parent document,
there not a single child document and we are adding new child document.
Following are the steps to reproduce the issue.

1. Add Child document to an existing parent document which is not having
empty child document.
2. Once the child document is added with delta indexing, try to modify the
parent document and run delta indexing again
3. After the delta indexing is completed, I can see the modified child
documents showing in Solr DIH page in debug mode. But the it is not getting
updated in Solr collection.

I am using data config as below as below.

  

  
  
  
  


  
 


  

  


-- 
Regards,

Aniket S. Khare

Using solr(cloud) as source-of-truth for data (with no backing external db)

[Dorian Hoxha]

Hi,

Anyone use solr for source-of-data with no `normal` db (of course with
normal backups/replication) ?

Are there any drawbacks ?

Thank You

compilation error

[Midas A]

gettting following error while compiling .
 .
org.apache.avro#avro;1.7.5: configuration not found in
org.apache.avro#avro;1.7.5: 'master'. It was required from
org.apache.solr#morphlines-core;


and not able to resolve . please help in resolving .

Re: compilation error

[Midas A]

Please reply?

On Thu, Nov 17, 2016 at 11:31 AM, Midas A  wrote:

> gettting following error while compiling .
>  .
> org.apache.avro#avro;1.7.5: configuration not found in
> org.apache.avro#avro;1.7.5: 'master'. It was required from
> org.apache.solr#morphlines-core;
>
>
> and not able to resolve . please help in resolving .
>

Solr filesystems: btrfs, xfs? Performance, stability, config...

[Will Moy]

Hi all

Does anyone have any advice or experience on using btrfs or xfs for Solr?

We've hit inode limits in ext (not because of Solr itself) and are
wondering about using something else.

I'm curious whether btrfs is stable enough, what configuration to use,
whether one is better than the other, and whether there are any benchmarks
available.

Thanks

Will


Full Fact. A registered charity (no. 1158683) and a non-profit company (no.
6975984) limited by guarantee and registered in England and Wales. 9
Warwick Court, London WC1R 5DJ.

Re: Search opening hours

[O. Klein]

Above implementation was too slow, so wondering if Solr 6 with all its new
features provides a better solution to tackle operating hours. Especially
dealing with different timezones.

Any thoughts?





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Search-opening-hours-tp4225250p4306073.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr Start-Up Error

[Ashish Kumar9]

Hi 

I am facing below exception issue when starting Solr in standalone mode on 
my environment 

Solr Version : 6.2.0 

I downloaded solr and built the source successfully it got built . During 
executing I first tried to run the built package and I got below exception 


=
[root@sys-77402 solr]# bin/solr start -f
Starting Solr on port 8983 from 
/tempdisk/software/mhahadoopdistro/solr-6.2.0/solr/server
The stack size specified is too small, Specify at least 328k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
[root@sys-77402 solr]# 
=

Then I thought this is generic java issue . and I specified the flag to 
bump up the memory available for jvm stack size 

#export _JAVA_OPTIONS="-Xss1g"

But when I further tried to start solr , I got below exception 

Starting Solr on port 8983 from 
/tempdisk/software/mhahadoopdistro/solr-6.2.0/solr/server

Picked up _JAVA_OPTIONS: -Xss1g -Xms5g -Xmx8g -Xrs
2016-11-16 06:26:51.614:INFO::main: Logging initialized @480ms
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
at org.eclipse.jetty.start.Main.start(Main.java:457)
at org.eclipse.jetty.start.Main.main(Main.java:75)
Caused by: java.lang.StackOverflowError
at 
java.util.concurrent.ConcurrentHashMap.tabAt(ConcurrentHashMap.java:755)
at 
java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1018)
at 
java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535)
at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463)
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at 

Re:NPE when executing clustering query search

[Christine Poerschke (BLOOMBERG/ LONDON)]

Hi Tim,

Thanks for reporting this. The (just created) 
https://issues.apache.org/jira/browse/SOLR-9775 issue and associated pull 
request sound related to this.

Regards,

Christine

- Original Message -
From: solr-user@lucene.apache.org
To: solr-user@lucene.apache.org
At: 03/22/16 14:49:20

Hi everyone,

I am trying to execute a clustering query to my single-core master-slave
solr setup and it is returning a NullPointerException.  I checked the line
in the source code where it is being thrown, and it looks like the null
object is some sort of 'filt' object, which doesn't make sense.  Below is
the query, my schema, solrconfig, and the exception.  If anyone could
please help that would be great!

Thank you!

QUERY:

1510649 [qtp1855032000-20] INFO  org.apache.solr.core.SolrCore  û
[collection1] webapp=/solr
path=/clustering
params{
mlt.minwl=3&
mlt.boost=true&
mlt.fl=textpropertymlt&
sort=score+desc&
carrot.snippet=impnoteplain&
mlt.mintf=1&
qf=concept_name&
mlt.interestingTerms=details&
wt=javabin&
clustering.engine=lingo&
version=2&
rows=500&
mlt.mindf=2&
debugQuery=true&
fl=id,concept_name,impnoteplain&
start=0&
q=id:567065dc658089be9f5c2c0d5670653d658089be9f5c2ae2&
carrot.title=concept_name&
clustering.results=true&
qt=/clustering&
fq=storeid:5670653d658089be9f5c2ae2&
fq={!edismax+v%3D''+qf%3D'textpropertymlt'+mm%3D'2<40%25'}=id=true}
status=500 QTime=217

ERROR:

1510697 [qtp1855032000-20] ERROR org.apache.solr.servlet.SolrDispatchFilter
 û null:java.lang.NullPointerException
at
org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1416)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:511)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:235)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:291)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2006)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:204)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)


SCHEMA.XML:

   
   
   
   

   
   

   
   
   
   
   
   

   


   
   

   

   
   

   
   
   
   c

   
   
   

   



   

   

   
   

   
   

   

   

Re: Solr Start-Up Error

[Shawn Heisey]

On 11/16/2016 1:25 AM, Ashish Kumar9 wrote:
> Solr Version : 6.2.0 
>
> I downloaded solr and built the source successfully it got built . During 
> executing I first tried to run the built package and I got below exception 

> Then I thought this is generic java issue . and I specified the flag to 
> bump up the memory available for jvm stack size 
>
> #export _JAVA_OPTIONS="-Xss1g"
>
> But when I further tried to start solr , I got below exception 
>
> Starting Solr on port 8983 from 
> /tempdisk/software/mhahadoopdistro/solr-6.2.0/solr/server
>
> Picked up _JAVA_OPTIONS: -Xss1g -Xms5g -Xmx8g -Xrs
> 2016-11-16 06:26:51.614:INFO::main: Logging initialized @480ms
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
> at org.eclipse.jetty.start.Main.start(Main.java:457)
> at org.eclipse.jetty.start.Main.main(Main.java:75)
> Caused by: java.lang.StackOverflowError

Some initial questions:  Why are you building Solr?  If the source is
unmodified, the binary release should be identical in function. Why not
just download the binary and use it?  Why are you using 6.2.0, when
there is a 6.2.1 bugfix release that's been out for nearly a month and
fixes at least 15 bugs in 6.2.0?

The stacktrace for the second problem does not contain a single Apache
class.  It's all native Java and Jetty.  It complains about running out
of stack space while loading classes.

Solr's scripts default the stack size to 256kb.  Yes, that's kilobytes,
not even megabytes. This is even smaller than Java 8 itself will use for
the stack size by default (320kb for 32-bit, 1mb for 64-bit), and Solr's
default is plenty large enough for an unmodified Solr to start with no
problems.  By setting the stack size to 1GB, it is 4096 times as large
as what Solr chooses by default (translation -- really really large),
but you're still blowing the stack.  That's a LOT of excess information
being loaded into memory.

Something about your install has been modified, so the first thing to do
is try an unmodified Solr.  If that works, then you would need to track
down the author of any modifications that you made, and ask them for
help.  If an unmodified download of the Solr binary install doesn't
work, then there's probably something about your environment that's bad.

Thanks,
Shawn

Solr Logs Info

[Mugeesh Husain]

In every day igot the below Info in solr logs files

2016-11-16 00:02:04.476 INFO  (qtp531008412-14) [   ]
o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/info/system
params={wt=json} status=0 QTime=38 


what does its mean ?




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Logs-Info-tp4306116.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: empty strings outputting to numeric field types

[John Blythe]

Ahh, makes sense. Thanks for the clarity, Hoss.

Best-

-- 
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | j...@curvolabs.com
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

On Tue, Nov 15, 2016 at 5:05 PM, Chris Hostetter 
wrote:

>
> : fields storing dollar values as tdouble. they don't always exist in the
> : outputted rows, however, at which point they throw an error and fail at
> : indexing because the field is seen as an empty string (the log message:
> str
> : = '').
> :
> : for now i've gotten around this by skipping out of any output for that
> : field in those cases, but wanted to know what the best method for
>
> Strictly speaking Solr isn't complaining because you give it an "empty
> string" it's complaining because you give it a string which can not be
> legally parsed as a double (or int, or float, etc...)
>
> Fixing your client to only send Solr valid numeric values, or no value
> when that's what you want for a given document, it what i would conider
> the most correct solution -- but if you want solr to ignore strings that
> aren't valid numeric values, that's what things like the
> RemoveBlankFieldUpdateProcessorFactory are for...
>
> https://lucene.apache.org/solr/6_3_0/solr-core/org/
> apache/solr/update/processor/RemoveBlankFieldUpdateProcessorFactory.html
>
> you can configure things like TrimFieldUpdateProcessorFactory and
> RegexReplaceProcessorFactory to pre-process string values to ignore
> whitespace or non decimal characters, etc... before they make it to the
> RemoveBlankFieldUpdateProcessorFactory.
>
>
>
> -Hoss
> http://www.lucidworks.com/
>

Apache Solr Reference Guide for 6.3 released

[Cassandra Targett]

The Lucene PMC is pleased to announce that the Solr Reference Guide
for Solr 6.3 has been released.

This 736-page PDF is the definitive guide to using Apache Solr, the
blazing fast search server built on Apache Lucene. The Guide can be
downloaded from:

https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide/apache-solr-ref-guide-6.3.pdf

Cassandra

Re: Basic Auth for Solr Streaming Expressions

[Kevin Risden]

Was a JIRA ever created for this? I couldn't find it searching.

One that is semi related is SOLR-8213 for SolrJ JDBC auth.

Kevin Risden

On Wed, Nov 9, 2016 at 8:25 PM, Joel Bernstein  wrote:

> Thanks for digging into this, let's create a jira ticket for this.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Nov 9, 2016 at 6:23 PM, sandeep mukherjee <
> wiredcit...@yahoo.com.invalid> wrote:
>
> > I have more progress since my last mail. I figured out that  in the
> > StreamContext object there is a way to set the SolrClientCache object
> which
> > keep reference to all the CloudSolrClient where I can set a reference to
> > HttpClient which sets the Basic Auth header. However the problem is,
> inside
> > the SolrClientCache there is no way to set your own version of
> > CloudSolrClient with BasicAuth enabled. Unfortunately, SolrClientCache
> has
> > no set method which takes a CloudSolrClient object.
> > So long story short we need an API in SolrClientCache to
> > accept CloudSolrClient object from user.
> > Please let me know if there is a better way to enable Basic Auth when
> > using StreamFactory as mentioned in my previous email.
> > Thanks much,Sandeep
> >
> > On Wednesday, November 9, 2016 11:44 AM, sandeep mukherjee
> >  wrote:
> >
> >
> >  Hello everyone,
> > I trying to find the documentation for Basic Auth plugin for Solr
> > Streaming expressions. But I'm not able to find it in the documentation
> > anywhere. Could you please point me in right direction of how to enable
> > Basic auth for Solr Streams?
> > I'm creating StreamFactory as follows: I wonder how and where can I
> > specify Basic Auth username and password
> > @Bean
> > public StreamFactory streamFactory() {
> > SolrConfig solrConfig = ConfigManager.getNamedConfig("solr",
> > SolrConfig.class);
> >
> > return new StreamFactory().withDefaultZkHost(solrConfig.
> > getConnectString())
> > .withFunctionName("gatherNodes", GatherNodesStream.class);
> > }
> >
> >
> >
>

Re: Basic Auth for Solr Streaming Expressions

[sandeep mukherjee]

 blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px 
#715FFA solid !important; padding-left:1ex !important; background-color:white 
!important; }  Nope never got past the login screen. Will create one today.


Sent from Yahoo Mail for iPhone


On Wednesday, November 16, 2016, 8:17 AM, Kevin Risden 
 wrote:

Was a JIRA ever created for this? I couldn't find it searching.

One that is semi related is SOLR-8213 for SolrJ JDBC auth.

Kevin Risden

On Wed, Nov 9, 2016 at 8:25 PM, Joel Bernstein  wrote:

> Thanks for digging into this, let's create a jira ticket for this.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Nov 9, 2016 at 6:23 PM, sandeep mukherjee <
> wiredcit...@yahoo.com.invalid> wrote:
>
> > I have more progress since my last mail. I figured out that  in the
> > StreamContext object there is a way to set the SolrClientCache object
> which
> > keep reference to all the CloudSolrClient where I can set a reference to
> > HttpClient which sets the Basic Auth header. However the problem is,
> inside
> > the SolrClientCache there is no way to set your own version of
> > CloudSolrClient with BasicAuth enabled. Unfortunately, SolrClientCache
> has
> > no set method which takes a CloudSolrClient object.
> > So long story short we need an API in SolrClientCache to
> > accept CloudSolrClient object from user.
> > Please let me know if there is a better way to enable Basic Auth when
> > using StreamFactory as mentioned in my previous email.
> > Thanks much,Sandeep
> >
> >    On Wednesday, November 9, 2016 11:44 AM, sandeep mukherjee
> >  wrote:
> >
> >
> >  Hello everyone,
> > I trying to find the documentation for Basic Auth plugin for Solr
> > Streaming expressions. But I'm not able to find it in the documentation
> > anywhere. Could you please point me in right direction of how to enable
> > Basic auth for Solr Streams?
> > I'm creating StreamFactory as follows: I wonder how and where can I
> > specify Basic Auth username and password
> > @Bean
> > public StreamFactory streamFactory() {
> >    SolrConfig solrConfig = ConfigManager.getNamedConfig("solr",
> > SolrConfig.class);
> >
> >    return new StreamFactory().withDefaultZkHost(solrConfig.
> > getConnectString())
> >            .withFunctionName("gatherNodes", GatherNodesStream.class);
> > }
> >
> >
> >
>

 

Re: book for Solr 3.4?

[Dorian Hoxha]

@HelponR
Curious why you're interested in an old version ?

On Tue, Nov 15, 2016 at 11:43 PM, HelponR  wrote:

> Thank you. Just found one here https://wiki.apache.org/solr/SolrResources
>
> "Apache Solr 3 Enterprise Search Server
>  by David Smiley and Eric
> Pugh. This is the 2nd edition of the first book, published by Packt.
> Essential reading for developers, this book covers nearly every feature up
> thru Solr 3.4. "
>
>
> On Tue, Nov 15, 2016 at 2:15 PM, Deeksha Sharma 
> wrote:
>
> > BTW its Apache Solr 4 Cookbook
> > 
> > From: Deeksha Sharma 
> > Sent: Tuesday, November 15, 2016 2:06 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: book for Solr 3.4?
> >
> > Apache solr cookbook will definitely help you get started. This is in
> > addition to the Apache Solr official documentation.
> >
> >
> > Thanks
> > Deeksha
> > 
> > From: HelponR 
> > Sent: Tuesday, November 15, 2016 2:03 PM
> > To: solr-user@lucene.apache.org
> > Subject: book for Solr 3.4?
> >
> > Hello!
> >
> > Is there a good book for Solr 3.4? The "Solr in Action" is for 4.4.
> >
> > googling did not help:(
> >
> > Thanks!
> >
>

potential issue/bug reported by Jetty team? was: Re: Change Solr contextPath =“/“ ?

[matthew grisius]

I can now deploy my app that uses Solr via Jetty similar to my previous 
tomcat/tomee+ stack war deployment thanks to the Jetty team (Simone Bordet).
It turned out to be very straightforward.
Per his suggestion I commented out the RewriteHandler def and reference in 
$solr/server/etc/jetty.xml.
Then I add standard servlet or static content at “/“ in server/etc/ dir:

CLJServlet.xml:


http://www.eclipse.org/jetty/configure_9_0.dtd 
">

  
  
  
  
  /webapps/CLJServlet
  /etc/webdefault.xml


foo.xml:


http://www.eclipse.org/jetty/configure_9_0.dtd 
">

  
  
  
  
  /webapps/foo
  

   

  


I did report an interesting Jetty warning that tomcat/tomee+ did not catch for 
my servlet:
Log4j (org.slf4j.impl.Log4jLoggerFactory) warnings: {/CLJServlet} has uncovered 
http methods for path: /

The Jetty team (Jan Bartel) responded back and suggested a fix and suggested 
that I notify the Solr team, here is his reply in case it might help Solr Team:

Looks like a little bug on solr.  In jetty 9.1.something we changed the 
definition of the webdefault.xml file to avoid the "Uncovered http methods" 
warning. This is related to the security-constraint for the TRACE method.  We 
used to have:

  

  Disable TRACE
  /
  TRACE


  

Which meant that according to the servlet spec there were indeed uncovered 
methods.

So we changed it to this couplet instead:

  

  Disable TRACE
  /
  TRACE


  
  

  Enable everything but TRACE
  /
  TRACE

  

However, I notice that solr has the old definition in their etc/webdefault.xml 
file, and they have chosen to add the extra definition only to the web.xml file 
of their solr-webapp.

So the easiest thing for you to do is:

1. copy the extra security-constraint into the etc/webdefault.xml file so it 
applies to all webapps
2. let solr know :)

> On Nov 7, 2016, at 12:14 PM, matthew grisius  
> wrote:
> 
> I previously asked this question and did not receive any suggestions.
> I also asked on Jetty-Users and one person suggested I ask again here so I 
> thought it might be worth it to ask again.
> 
> Solr is deployed to “/solr” and is then mapped to “/“, e.g. 
> http://localhost:8983/  gets mapped to 
> http://localhost:8983/solr/# .
> I want to put a different servlet at “/“ while preserving Solr functionality 
> at “/solr”.
> The reason for this is that I previously deployed my app with Solr on 
> tomcat/tomee via war file and now want to use a current Solr release using 
> Solr/Jetty.
> 
> I thought the way to accomplish that was to re-assign  the "root context", 
> e.g. contextPath =“/“ but cannot find a way to do this with Solr/Jetty.
> 
> My unsuccessful attempts included:
> 
> - edit contexts/solr-jetty-context.xml (contextPath, add resourceBase, etc.)
> - unmap solr rewrite rule to pass thru . . .
> - edit other '/contexts/' . . .
> - edit etc/webdefault.xml . . .
> - bin/solr does not appear to affect "/"
> - org.apache.solr.util.SolrCLI does not appear to affect "/"
> 
> Perhaps I’m not trying the right combination of things in the right order to 
> solve the issue, but I’ve run out of simple ideas to try.
> 
> Does anyone have any other suggestions?
> Thank you, any help would be appreciated!
> 
> -matt.

Re: Solr Logs Info

[Mugeesh Husain]

Erik,

But I didn't open admin GUI for long time but its showing me this request on
my logs.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Logs-Info-tp4306116p4306154.html
Sent from the Solr - User mailing list archive at Nabble.com.

Adding retries for making calls to solr using solrj

[pdj xyz]

Hi,

We are seeing transient Connection reset in our custom solr client(a
wrapper around solrj). We want to add retries to all methods that we are
currently using so that the we are able to upload successfully. However,
I'm not sure if there's any relevant documentation on which methods are
idempotent and which aren't.

Our use case - We have a single solr host. We aren't using solr cloud or
anything fancy,

We want to upload an index to Solr host. To do that, we first:
1) Disable replication
2) delete old index
3) upload new index
4) commit the changes (rollback if there's an exception)
5) run a solr query and perform some validations
6) run /admin/luke and perform some validation.
7) enable replication

We're currently thinking it should be OK to retry each of these 6
requests(atleast for Socket Exceptions), but would like
guidance/confirmation. Any documentation on this would be really helpful.

Thanks

-- 
Pranay

Re: Change Solr contextPath =“/“ ?

[Christian Ortner]

Hi Matthew,

your problem sounds like you want to run something alongside Solr, that
probably uses Solr. Since current versions of Solr basically require you to
go over HTTP, you could deploy the thing you would like to run in the root
context in a separate application container that accesses Solr over HTTP.

Alternatively, if the path is the only issue, you could just put a reverse
proxy in front of your application server and re-map the path to you
context that sits beside Solr.

Making changes like the ones you are suggesting are difficult in current
versions of Solr, where everything is coupled tightly to Jetty, so I hope
my suggestions can inspire you to find the perfect solution for this
problem.

Best regards,
Chris

On Mon, Nov 7, 2016 at 6:14 PM, matthew grisius  wrote:

> I previously asked this question and did not receive any suggestions.
> I also asked on Jetty-Users and one person suggested I ask again here so I
> thought it might be worth it to ask again.
>
> Solr is deployed to “/solr” and is then mapped to “/“, e.g.
> http://localhost:8983/  gets mapped to
> http://localhost:8983/solr/# .
> I want to put a different servlet at “/“ while preserving Solr
> functionality at “/solr”.
> The reason for this is that I previously deployed my app with Solr on
> tomcat/tomee via war file and now want to use a current Solr release using
> Solr/Jetty.
>
> I thought the way to accomplish that was to re-assign  the "root context",
> e.g. contextPath =“/“ but cannot find a way to do this with Solr/Jetty.
>
> My unsuccessful attempts included:
>
> - edit contexts/solr-jetty-context.xml (contextPath, add resourceBase,
> etc.)
> - unmap solr rewrite rule to pass thru . . .
> - edit other '/contexts/' . . .
> - edit etc/webdefault.xml . . .
> - bin/solr does not appear to affect "/"
> - org.apache.solr.util.SolrCLI does not appear to affect "/"
>
> Perhaps I’m not trying the right combination of things in the right order
> to solve the issue, but I’ve run out of simple ideas to try.
>
> Does anyone have any other suggestions?
> Thank you, any help would be appreciated!
>
> -matt.

Handling ampersands in searches.

[Callum Lamb]

I'm having an issue where searches that contain ampersands aren't being
handled correctly. I need them to be dropped at index time *AND* query
time. When documents come in and are indexed the ampersands are
successfully dropped when they go into my stemmed field (When I facet on
the stemmed field they aren't in the list), but when I actually search with
a term containing an ampersand, I get no results.

E.g. I search for the string "light fit" or "light and fit" then I get
results, but when I search for "light & fit" I get none. Even though the
SnowballPorterFilterFactory should be dropping it at query time like it
does for the "and" and all 3 queries *should* be equivalent.

I've tried adding a synonym such that shows in
my _schema_analysis_synonyms_default.json (I only have one default file) in
both this form and its inverse as well:

"and":[

  "&",
  "and"],


I've also tried adding the StopWord filter to my fieldtype with & in the
stopwords (though this shouldn't be necessary because the SnowBallPorter
should be dropping it anyway) and it still doesn't work.

Is there some kind of special handling I need for ampersands? I'm thinking
that Solr must be interpreting it as some kind of operator and I need to
tell Solr that it's actually literal text so the SnowBallPorter knows to
drop it. Using backslashes or url encoding instead doesn't work though.
Does anyone have any ideas?

I can obviously just remove any ampersands from the q before I submit the
query to Solr and get the correct results, so this is not a game breaking
problem, but i'm more curious to *why* this is happening and how to fix it
correctly.

Cheers,

Callum.

Extra info:

I'm using Solr 5.5.2 in cloud mode.

The q in the queries is specified like this and are parsed the following
way:

"rawquerystring":"stemmed_description:light & fit", "querystring":"
stemmed_description:light & fit", "parsedquery":"(+(+stemmed_description:light
+DisjunctionMaxQuery((stemmed_description:&)) +DisjunctionMaxQuery((
stemmed_description:fit/no_coord", "parsedquery_toString":"+(+
stemmed_description:light +(stemmed_description:&) +(stemmed_description
:fit))",

I have a stemmed field defined in my schema (schema version 1.5) defined
like this:



with a field type defined like this:



  







  
  







  


-- 

Mintel Group Ltd | 11 Pilgrim Street | London | EC4V 6RN
Registered in England: Number 1475918. | VAT Number: GB 232 9342 72

Contact details for our other offices can be found at 
http://www.mintel.com/office-locations.

This email and any attachments may include content that is confidential, 
privileged 
or otherwise protected under applicable law. Unauthorised disclosure, 
copying, distribution 
or use of the contents is prohibited and may be unlawful. If you have 
received this email in error,
including without appropriate authorisation, then please reply to the 
sender about the error 
and delete this email and any attachments.

Re: Solr Logs Info

[Erik Hatcher]

Probably means you went to the admin UI that made that request?   
/admin/info/system is a special request handler that provides system 
information.

Erik

> On Nov 16, 2016, at 9:35 AM, Mugeesh Husain  wrote:
> 
> In every day igot the below Info in solr logs files
> 
> 2016-11-16 00:02:04.476 INFO  (qtp531008412-14) [   ]
> o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/info/system
> params={wt=json} status=0 QTime=38 
> 
> 
> what does its mean ?
> 
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-Logs-Info-tp4306116.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Error "unexpected docvalues type NUMERIC for field" using rord() function query on single valued int field

[Jaco de Vroed]

Hello Solr users,

I’m running into an error situation using Solr 5.3.3. The case is as follows. 
In my schema, I have a field with a definition like this:



That field is used in function queries for boosting purposes, using the rord() 
function. We’re coming from Solr 4, not using docValues for that field, and now 
moving to Solr 5, using docValues. Now, this is causing a problem. When doing 
this:


http://localhost:8983/solr/core1/select?q=*:*=ID,recip(rord(PublicationDate),0.15,300,10)
 


The following error is given: "unexpected docvalues type NUMERIC for field 
'PublicationDate' (expected one of [SORTED, SORTED_SET]). Use UninvertingReader 
or index with docvalues” (full stack trace below).

This does not happen when the field is changed to be multiValued, but I don’t 
want to change that at this point (and I noticed that changing from single 
valued to multivalued, then attempting to post the document again also results 
in an error related to docvalues type, but that could be the topic of another 
mail I guess). This is now blocking our long desired upgrade to Solr 5. We 
initially tried upgrading without docValues, but performance was completely 
killed because of our function query based ranking stuff, so we decide to use 
docValues.

To me, this seems a bug. I’ve tried finding something in Solr’s JIRA, the exact 
same error is in https://issues.apache.org/jira/browse/SOLR-7495 
, but that is a different case.

I can create a JIRA issue for this of course, but first wanted to throw this at 
the mailing list to see if there’s any insights that can be shared.

Thanks a lot in advance, bye, 

Jaco..

unexpected docvalues type NUMERIC for field 'PublicationDate' (expected one of 
[SORTED, SORTED_SET]). Use UninvertingReader or index with docvalues.
java.lang.IllegalStateException: unexpected docvalues type NUMERIC for field 
'PublicationDate' (expected one of [SORTED, SORTED_SET]). Use UninvertingReader 
or index with docvalues.
at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
at org.apache.lucene.index.DocValues.getSortedSet(DocValues.java:306)
at 
org.apache.solr.search.function.ReverseOrdFieldSource.getValues(ReverseOrdFieldSource.java:98)
at 
org.apache.lucene.queries.function.valuesource.ReciprocalFloatFunction.getValues(ReciprocalFloatFunction.java:64)
at 
org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95)
at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:160)
at 
org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:246)
at 
org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:151)
at org.apache.solr.response.XMLWriter.writeResponse(XMLWriter.java:113)
at 
org.apache.solr.response.XMLResponseWriter.write(XMLResponseWriter.java:39)
at 
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:52)
at 
org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 

Re: Error "unexpected docvalues type NUMERIC for field" using rord() function query on single valued int field

[Jaco de Vroed]

Hi,

I made a typo. The Solr version number in which this error occurs is 5.5.3.
I also checked 6.3.0, same problem.

Thanks, bye,

Jaco.

On 16 November 2016 at 17:39, Jaco de Vroed  wrote:

> Hello Solr users,
>
> I’m running into an error situation using Solr 5.3.3. The case is as
> follows. In my schema, I have a field with a definition like this:
>
>  positionIncrementGap="0”/>
> ….
>  docValues="true" />
>
> That field is used in function queries for boosting purposes, using the
> rord() function. We’re coming from Solr 4, not using docValues for that
> field, and now moving to Solr 5, using docValues. Now, this is causing a
> problem. When doing this:
>
> http://localhost:8983/solr/core1/select?q=*:*=ID,
> recip(rord(PublicationDate),0.15,300,10)
>
> The following error is given: "*unexpected docvalues type NUMERIC for
> field 'PublicationDate' (expected one of [SORTED, SORTED_SET]). Use
> UninvertingReader or index with docvalues*” (full stack trace below).
>
> This does not happen when the field is changed to be multiValued, but I
> don’t want to change that at this point (and I noticed that changing from
> single valued to multivalued, then attempting to post the document again
> also results in an error related to docvalues type, but that could be the
> topic of another mail I guess). This is now blocking our long desired
> upgrade to Solr 5. We initially tried upgrading without docValues, but
> performance was completely killed because of our function query based
> ranking stuff, so we decide to use docValues.
>
> To me, this seems a bug. I’ve tried finding something in Solr’s JIRA, the
> exact same error is in https://issues.apache.org/jira/browse/SOLR-7495,
> but that is a different case.
>
> I can create a JIRA issue for this of course, but first wanted to throw
> this at the mailing list to see if there’s any insights that can be shared.
>
> Thanks a lot in advance, bye,
>
> Jaco..
>
> unexpected docvalues type NUMERIC for field 'PublicationDate' (expected
> one of [SORTED, SORTED_SET]). Use UninvertingReader or index with docvalues.
> java.lang.IllegalStateException: unexpected docvalues type NUMERIC for
> field 'PublicationDate' (expected one of [SORTED, SORTED_SET]). Use
> UninvertingReader or index with docvalues.
> at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
> at org.apache.lucene.index.DocValues.getSortedSet(DocValues.java:306)
> at org.apache.solr.search.function.ReverseOrdFieldSource.getValues(
> ReverseOrdFieldSource.java:98)
> at org.apache.lucene.queries.function.valuesource.ReciprocalFloatFunction.
> getValues(ReciprocalFloatFunction.java:64)
> at org.apache.solr.response.transform.ValueSourceAugmenter.transform(
> ValueSourceAugmenter.java:95)
> at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:160)
> at org.apache.solr.response.TextResponseWriter.writeDocuments(
> TextResponseWriter.java:246)
> at org.apache.solr.response.TextResponseWriter.writeVal(
> TextResponseWriter.java:151)
> at org.apache.solr.response.XMLWriter.writeResponse(XMLWriter.java:113)
> at org.apache.solr.response.XMLResponseWriter.write(
> XMLResponseWriter.java:39)
> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(
> QueryResponseWriterUtil.java:52)
> at org.apache.solr.servlet.HttpSolrCall.writeResponse(
> HttpSolrCall.java:728)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:257)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:208)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1652)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:585)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:577)
> at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:223)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:515)
> at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1061)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:215)
> at org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:110)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at org.eclipse.jetty.server.HttpConnection.onFillable(
> 

Detecting schema errors while adding documents

[Mike Thomsen]

We're stuck on Solr 4.10.3 (Cloudera bundle). Is there any way to detect
with SolrJ when a document added to the index violated the schema? All we
see when we look at the stacktrace for the SolrException that comes back is
that it contains messages about an IOException when talking to the solr
nodes. Solr is up and running, and the documents are only invalid because I
added a Java statement to make a field invalid for testing purposes. When I
remove that statement, the indexing happens just fine.

Any way to do this? I seem to recall that at least in newer versions of
Solr it would tell you more about the specific error.

Thanks,

Mike

Re: Solr Logs Info

[Erik Hatcher]

What IP address is that request coming from?   Perhaps you can track it down 
from there.


> On Nov 16, 2016, at 12:25 PM, Mugeesh Husain  wrote:
> 
> Erik,
> 
> But I didn't open admin GUI for long time but its showing me this request on
> my logs.
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-Logs-Info-tp4306116p4306154.html
> Sent from the Solr - User mailing list archive at Nabble.com.