Re: Search and Entity structure

[v vijith]

Hi,

Dear All,

Apologize for lengthy email 

SOLR Version: 4

Im a newbie to SOLR and have gone through tutorial but could not get a
solution. The below requirement doesnt seem to be impossible but I
think Im missing the obvious.

In my RDBMS, there is a Qualification table and an Employee table. An
employee can have many qualifications. The qualification can have
following attributes - GradeName and Grade. The search using sql query
to achieve my requirement is as below

select * from qualification a, employee b where a.empid= b.empid and
a.gradename='MBA' and b.grade='A';

This will return me the employee along with the dept who has the grade
as MBA and has grade of A.

Employee: 2 records
-
Empid: 1
Name: John
Location: California

Qualifications:
Gradeid: 1
Empid: 1
Name: MBA
Grade: B

Gradeid: 2
Empid: 1
Name: LEAD
Grade: A


Empid: 2
Name: George
Location: Nevada

Qualifications:
Gradeid: 3
Empid: 2
Name: MBA
Grade: A

Gradeid: 4
Empid: 2
Name: Graduate
Grade: C
-
With the above query, I get the following record as the output
Empid: 2
Name: George
Grade Name: MBA
Grade: A

Note that, I dont get Emp with empid: 1 as the qualification is MBA
and has grade B only.

I would like to get this done through SOLR.

Note that, I have simplified my problem using emp and qualification
tables so that its easy to understand.

Once I configured the Schema.xml and data-config.xml, I was able to
index the employee and qualification and the
I have an entity structure generated by SOLR like this


 John<./str>
California 


 MBA
 LEAD



 A
 B




 George<./str>
Nevada


 MBA
 Graduate



 A
 C



As you see in the above structure, the relation between qualifications
are lost. The linkage between MBA and  its corresponding Grade is
lost. So IF I do a search with Grade Name MBA and Grade as A, both the
docs will be returned.  This is incorrect data for me.

Is there a way by which I can get relation between the child entity
elements. In short how do I get the result as shown in the SQL above?

The answer probably is in the document or entity that is getting generated.

If the entity is as below, it might fix the same


 George<./str>
Nevada


 MBA


 A



 George<./str>
Nevada


 graduate


 C


Then, when I search, it will show one doc only.

Thanks
Viju


Thanks
Viju

On Thu, Oct 25, 2012 at 8:18 PM, v vijith  wrote:
> Dear All,
>
> Apologize for lengthy email 
>
> SOLR Version: 4
>
> Im a newbie to SOLR and have gone through tutorial but could not get a
> solution. The below requirement doesnt seem to be impossible but I
> think Im missing the obvious.
>
> In my RDBMS, there is a Qualification table and an Employee table. An
> employee can have many qualifications. The qualification can have
> following attributes - GradeName and Grade. The search using sql query
> to achieve my requirement is as below
>
> select * from qualification a, employee b where a.empid= b.empid and
> a.gradename='MBA' and b.grade='A';
>
> This will return me the employee along with the dept who has the grade
> as MBA and has grade of A.
>
> Employee: 2 records
> -
> Empid: 1
> Name: John
> Location: California
>
> Qualifications:
> Gradeid: 1
> Empid: 1
> Name: MBA
> Grade: B
>
> Gradeid: 2
> Empid: 1
> Name: LEAD
> Grade: A
> 
>
> Empid: 2
> Name: George
> Location: Nevada
>
> Qualifications:
> Gradeid: 3
> Empid: 2
> Name: MBA
> Grade: A
>
> Gradeid: 4
> Empid: 2
> Name: Graduate
> Grade: C
> -
> With the above query, I get the following record as the output
> Empid: 2
> Name: George
> Grade Name: MBA
> Grade: A
>
> Note that, I dont get Emp with empid: 1 as the qualification is MBA
> and has grade B only.
>
> I would like to get this done through SOLR.
>
> Note that, I have simplified my problem using emp and qualification
> tables so that its easy to understand.
>
> Once I configured the Schema.xml and data-config.xml, I was able to
> index the employee and qualification and the
> I have an entity structure generated by SOLR like this
>
> 
>  John<./str>
> California 
>
> 
>  MBA
>  LEAD
> 
>
> 
>  A
>  B
> 
> 
>
> 
>  George<./str>
> Nevada
>
> 
>  MBA
>  Graduate
> 
>
> 
>  A
>  C
> 
> 
>
> As you see in the above structure, the relation between qualifications
> are lost. The linkage between MBA and  its corresponding Grade is
> lost. So IF I do a search with Grade Name MBA and Grade as A, both the
> docs will be returned.  This is incorrect data for me.
>
> Is there a way by which I can get relation between the child entity
> elements. In short how do I get the result as shown in the SQL above?
>
> The answer probably is in the document or entity that is getting generated.
>
> If the entity is as below, it might fix the same
>
> 
>  George<./str>
> Nevada
>
> 
>  MBA
> 
> 
>  A
> 
>
> 
>  George<./str>
> Nevada
>
> 
>  graduate
> 
> 
>  C
> 
>
> Then, when I search, it will show one doc only.
>
> Thanks
> Viju

Re: DIH update?

[Gora Mohanty]

On 26 October 2012 08:51, Billy Newman  wrote:
> Any plans on adding update functionality to DIH?

What do you mean by "update functionality"?

Re-running an import with changed values for a
document with an existing ID will update values
in the Solr index.

If you mean adding new documents, please take
a look at delta import.

Regards,
Gora

DIH update?

[Billy Newman]

Any plans on adding update functionality to DIH?  

Re: Best way to commit data to Solr

[Tomás Fernández Löbbe]

If you don't need them searchable, you don't use Solr4 with transaction log
and it is a relatively fast process (so it's not a pain to run it all again
in case of some failure), I would just commit at the end. In any other
case, I would use autocommit.
Anyway, I'm not sure if you'll see much difference.

Tomás

On Thu, Oct 25, 2012 at 7:42 PM, adityab  wrote:

> Which is the best way to commit documents to Solr when publishing larger
> number of documents. for e.g around 10M to 20M documents. They are not
> expected to be searchable right away.
> We have Solr 3.5 implemented as Master/Repeater/Slave.
> Also investigating same for Solr 4.0 with same Master/Repeater/Slave
> config.
>
> 1. Commit it at the end when all documents are published.
> 2. Or Commit it after certain time interval or number of docs by using the
> autoCommit feature.
>
> Or any other better alternates.
>
> thanks
> AB
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Best-way-to-commit-data-to-Solr-tp4015921.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Field Grouping problem/question

[Tomás Fernández Löbbe]

By default you get the top result from each group (if you are not sorting
on a field specifically, then you should be getting the document with the
highest score per group).

This can be changed with the "group.limit" parameter. See
http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters

Tomás

On Thu, Oct 25, 2012 at 5:41 PM, Billy Newman  wrote:

> I am currently trying to use field grouping and I am not getting the
> results I am expecting.
>
> I have the following fields
>
> id
> type
> group
> animal_name
> animal_description
> mineral_name
> mineral_description
> description (copyfield from animal_description and mineral_description)
>
> &q=description:"Some Text"&group=true&group.field=group
>
> I get back results, however I do not get back all fields.  For some
> reason in the response I do no get back the mineral specific fields.
> Should I get back a combination of all fields from each group?
>
>  I.E. should I expect to get back all the fields from those I defined
> above?
>
> Also if I am supposed to get back all fields what happens to fields
> from each type that have different values but are stored in the same
> field (Sorry for the bad example)
>
> id=1234
> type=animal
> group=ONE
> animal_name=fox
> animal_description=Some Text
>
>
> id=5678
> type=mineral
> group=ONE
> mineral_name=gold
> mineral_description=Some Text
>
> So for the query above on "Some Text" I should get back 2 results but
> they should be grouped into one based on the fact that they live in
> group ONE.
> I would expect my results to come back with all fields collapsed into one
> group:
>
> id=? ( not sure what comes back here 1234 or 5678)
> type=? (not sure what comes back here animal or mineral)
> group=ONE
> animal_name=fox
> animal_description=Some Text
> mineral_name=gold
> mineral_description=Some Text
>
> however I am only getting back fields from one of the results, in this
> case only the animal stuff is coming back:
> id=1234
> type=animal or mineral
> group=ONE
> animal_name=fox
> animal_description=Some Text
>
>
> Any ideas?
>
> Billy
>

Re: throttle segment merging

[Tomás Fernández Löbbe]

>
> Is there way to set-up logging to output something when segment merging
> runs?
>
I think segment merging is logged when you enable infoStream logging (you
should see it commented in the solrconfig.xml)

>
> Can be segment merges throttled?
>

You can change when and how segments are merged with the merge policy,
maybe it's enough for you changing the initial settings (mergeFactor for
example)?

Tomás

throttle segment merging

[Radim Kolar]

I have problems with very low indexing speed as soon as core size grows 
over 15 GB. I suspect that it can be due io intensive segment merging.


Is there way to set-up logging to output something when segment merging 
runs?


Can be segment merges throttled?

Re: How to boost exact match?

[Samuel García Martínez]

whops! Jack got it faster :D

Sorry for "double" posting.

On Thu, Oct 25, 2012 at 11:41 PM, Jack Krupansky wrote:

> You may want to do a copyfield and have one field that is stemmed for
> casual matches at a lower boost while one is unstemmed and boosted higher.
>
> You could also do a copyfield to a non-tokenized field (TextField with
> KeywordTokenizer and lower case and trim filters) and give a match in that
> field a much higher boost.
>
>
> -- Jack Krupansky
>
> -Original Message- From: bbarani
> Sent: Thursday, October 25, 2012 5:34 PM
> To: solr-user@lucene.apache.org
> Subject: Re: How to boost exact match?
>
>
> Thanks a lot for your reply.
>
> Consider the below query
>
> ?q=data management&defType=edismax&qf=**name^100 text uid^0.3&fl=name
>
> I am trying to boost the name as much as possible, even then the results
> are
> in below order
>
> - 
>  MANAGER
>  
> - 
>  MANAGER
>  
> - 
>  MANAGERS
>  
> - 
>  ...AA DATA MANAGEMENT
>  
> - 
>  BA...AA DATA MANAGEMENT
>
> When I turned on the debug query, I figured out that the actual keyword
> passed is stemmed due to usage of PorterStemFilterFactory which actually
> splits the keyword in to data manag and passes that to search.
>
> I  am using PorterStemFilterFactory for both indexing / query.
>
> I will try to remove the porter stem from query and check if I am getting
> proper result.
>
> Thanks.
> BB
>
> Thanks,
> BB
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.**
> nabble.com/How-to-boost-exact-**match-tp4015862p4015915.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Un saludo,
Samuel García.

Re: How to boost exact match?

[Samuel García Martínez]

First of all, if you stem on indexing and you don't on query time, your
queries wont find anything valuable.

This is an easy approach. If you want full term (no stemmed) matches were
boosted you have to query like:

?q=data management&defType=edismax&qf=*name_without_stemming^200* name^100
text uid^0.3&fl=name

On schema.xml you have to copy "name" to "name_without_stemming"(indexed
and no stored) and don't use stemming on this field index and query
analyzer definition.

On Thu, Oct 25, 2012 at 11:34 PM, bbarani  wrote:

> Thanks a lot for your reply.
>
> Consider the below query
>
> ?q=data management&defType=edismax&qf=name^100 text uid^0.3&fl=name
>
> I am trying to boost the name as much as possible, even then the results
> are
> in below order
>
> - 
>   MANAGER
>   
> - 
>   MANAGER
>   
> - 
>   MANAGERS
>   
> - 
>   ...AA DATA MANAGEMENT
>   
> - 
>   BA...AA DATA MANAGEMENT
>
> When I turned on the debug query, I figured out that the actual keyword
> passed is stemmed due to usage of PorterStemFilterFactory which actually
> splits the keyword in to data manag and passes that to search.
>
> I  am using PorterStemFilterFactory for both indexing / query.
>
> I will try to remove the porter stem from query and check if I am getting
> proper result.
>
> Thanks.
> BB
>
> Thanks,
> BB
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/How-to-boost-exact-match-tp4015862p4015915.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Un saludo,
Samuel García.

Re: How to boost exact match?

[Jack Krupansky]

You may want to do a copyfield and have one field that is stemmed for casual 
matches at a lower boost while one is unstemmed and boosted higher.


You could also do a copyfield to a non-tokenized field (TextField with 
KeywordTokenizer and lower case and trim filters) and give a match in that 
field a much higher boost.


-- Jack Krupansky

-Original Message- 
From: bbarani

Sent: Thursday, October 25, 2012 5:34 PM
To: solr-user@lucene.apache.org
Subject: Re: How to boost exact match?

Thanks a lot for your reply.

Consider the below query

?q=data management&defType=edismax&qf=name^100 text uid^0.3&fl=name

I am trying to boost the name as much as possible, even then the results are
in below order

- 
 MANAGER
 
- 
 MANAGER
 
- 
 MANAGERS
 
- 
 ...AA DATA MANAGEMENT
 
- 
 BA...AA DATA MANAGEMENT

When I turned on the debug query, I figured out that the actual keyword
passed is stemmed due to usage of PorterStemFilterFactory which actually
splits the keyword in to data manag and passes that to search.

I  am using PorterStemFilterFactory for both indexing / query.

I will try to remove the porter stem from query and check if I am getting
proper result.

Thanks.
BB

Thanks,
BB




--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-exact-match-tp4015862p4015915.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: How to boost exact match?

[bbarani]

Thanks a lot for your reply.

Consider the below query

?q=data management&defType=edismax&qf=name^100 text uid^0.3&fl=name

I am trying to boost the name as much as possible, even then the results are
in below order

- 
  MANAGER 
  
- 
  MANAGER 
  
- 
  MANAGERS 
  
- 
  ...AA DATA MANAGEMENT 
  
- 
  BA...AA DATA MANAGEMENT 

When I turned on the debug query, I figured out that the actual keyword
passed is stemmed due to usage of PorterStemFilterFactory which actually
splits the keyword in to data manag and passes that to search.

I  am using PorterStemFilterFactory for both indexing / query.

I will try to remove the porter stem from query and check if I am getting
proper result.

Thanks.
BB

Thanks,
BB




--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-exact-match-tp4015862p4015915.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud and distributed search

[Tomás Fernández Löbbe]

It also provides high availability for indexing and searching.

On Thu, Oct 25, 2012 at 4:43 PM, Bill Au  wrote:

> So I guess one would use SolrCloud for the same reasons as distributed
> search:
>
> When an index becomes too large to fit on a single system, or when a single
> query takes too long to execute.
>
> Bill
>
> On Thu, Oct 25, 2012 at 3:38 PM, Shawn Heisey  wrote:
>
> > On 10/25/2012 1:29 PM, Bill Au wrote:
> >
> >> Is SolrCloud using distributed search behind the scene?  Does it have
> the
> >> same limitations (for example, doesn't support MoreLikeThis) distributed
> >> search has?
> >>
> >
> > Yes and yes.
> >
> >
>

Re: SolrCloud and distributed search

[Bill Au]

So I guess one would use SolrCloud for the same reasons as distributed
search:

When an index becomes too large to fit on a single system, or when a single
query takes too long to execute.

Bill

On Thu, Oct 25, 2012 at 3:38 PM, Shawn Heisey  wrote:

> On 10/25/2012 1:29 PM, Bill Au wrote:
>
>> Is SolrCloud using distributed search behind the scene?  Does it have the
>> same limitations (for example, doesn't support MoreLikeThis) distributed
>> search has?
>>
>
> Yes and yes.
>
>

Re: SolrCloud and distributed search

[Shawn Heisey]

On 10/25/2012 1:29 PM, Bill Au wrote:

Is SolrCloud using distributed search behind the scene?  Does it have the
same limitations (for example, doesn't support MoreLikeThis) distributed
search has?


Yes and yes.

Search and Entity structure

[v vijith]

Dear All,

Apologize for lengthy email 

SOLR Version: 4

Im a newbie to SOLR and have gone through tutorial but could not get a
solution. The below requirement doesnt seem to be impossible but I
think Im missing the obvious.

In my RDBMS, there is a Qualification table and an Employee table. An
employee can have many qualifications. The qualification can have
following attributes - GradeName and Grade. The search using sql query
to achieve my requirement is as below

select * from qualification a, employee b where a.empid= b.empid and
a.gradename='MBA' and b.grade='A';

This will return me the employee along with the dept who has the grade
as MBA and has grade of A.

Employee: 2 records
-
Empid: 1
Name: John
Location: California

Qualifications:
Gradeid: 1
Empid: 1
Name: MBA
Grade: B

Gradeid: 2
Empid: 1
Name: LEAD
Grade: A


Empid: 2
Name: George
Location: Nevada

Qualifications:
Gradeid: 3
Empid: 2
Name: MBA
Grade: A

Gradeid: 4
Empid: 2
Name: Graduate
Grade: C
-
With the above query, I get the following record as the output
Empid: 2
Name: George
Grade Name: MBA
Grade: A

Note that, I dont get Emp with empid: 1 as the qualification is MBA
and has grade B only.

I would like to get this done through SOLR.

Note that, I have simplified my problem using emp and qualification
tables so that its easy to understand.

Once I configured the Schema.xml and data-config.xml, I was able to
index the employee and qualification and the
I have an entity structure generated by SOLR like this


 John<./str>
California 


 MBA
 LEAD



 A
 B




 George<./str>
Nevada


 MBA
 Graduate



 A
 C



As you see in the above structure, the relation between qualifications
are lost. The linkage between MBA and  its corresponding Grade is
lost. So IF I do a search with Grade Name MBA and Grade as A, both the
docs will be returned.  This is incorrect data for me.

Is there a way by which I can get relation between the child entity
elements. In short how do I get the result as shown in the SQL above?

The answer probably is in the document or entity that is getting generated.

If the entity is as below, it might fix the same


 George<./str>
Nevada


 MBA


 A



 George<./str>
Nevada


 graduate


 C


Then, when I search, it will show one doc only.

Thanks
Viju

solr -autosuggest

[Sujatha Arun]

Hi,

A  few question on Solr Auto suggest below

Q1)I tried using  the  Index based Suggest functionality with  solr 3.6.1 ,
can I combine this with  file based boosting .Currently when I specify the
index field and the sourcelocation,the file in the source location is not
considered.
Is there any way both can be used?

Q2)I saw this line where it says "Currently implemented Lookups keep their
data in memory, so unlike spellchecker data, this data is discarded on core
reload and not available until you invoke the build command, either
explicitly or implicitly during a commit."I have used the wfst lookup  and
using the index based suggestion ,I suppose that this applies to only File
based suggestion? Is this correct?


Q3) if spellcheck.onlyMorePopular=true is selected: weights are treated as
"popularity" score ,Does this mean that this is based on frequency of words
 or is this based on ranking [tf * idf...ect] ?



Regards,
Sujatha

Re: Field in DataConfig not in Solr Schema

[Shawn Heisey]

On 10/25/2012 10:52 AM, Billy Newman wrote:

This is OK by me since I don't really want index that field anyways,
but I am wondering if I am doing something wrong.  Is there a way to
mark a column in the Data Config as 'not indexed'.  Or should I put a
field in my Solr Schema and mark it as indexed=false, stored=false?


For fields that I do not want to have in my index even though they are 
presented by my database to the dataimport handler, I use the following 
fieldtype:


indexed="false" multiValued="true" omitNorms="true"/>


Then under fields, I can do the following.  I probably don't need the 
indexed/stored parameters on the field because they are specified on the 
fieldType, but it isn't hurting anything to have them:


   

Thanks,
Shawn

Field in DataConfig not in Solr Schema

[Billy Newman]

I am using columns in my DataConfig specifically to grab them later as
a template.

i.e.




Where stuff is the field I want to index, but all stuff is just a
temporary column used such that I can get at it via the template
transformer.  'stuff' is in my schema 'allStuff' is not.

I get the following info message:
The field :allStuff present in DataConfig does not have a counterpart
in Solr Schema.

This is OK by me since I don't really want index that field anyways,
but I am wondering if I am doing something wrong.  Is there a way to
mark a column in the Data Config as 'not indexed'.  Or should I put a
field in my Solr Schema and mark it as indexed=false, stored=false?

Thanks,
Billy

Re: How to boost exact match?

[Jack Krupansky]

You neglected to show us your example queries and how you wanted to boost 
them. The simple answer is to supply multiple query terms with increasing 
level of detail (e.g., more terms in a quoted phrase) and apply a query 
boost that is higher for the more exact matches.


If you don't get expected results, please detail precisely which results 
were not as expected and exactly why you found them to be unexpected.


Add &debugQuery=true to your query request to see how each document is being 
scored in the "explain" section. Sometimes that will offer clues as to what 
to focus boosting on.


And finally, you may want to set a document boost at index time for some 
documents, or maybe define one or more extra fields with boost factors as 
their values at index time to be used in various query situations and then 
add a function query to your query to apply those document-specific boosts. 
Or, those fields could have simple booleans to trigger boost functions.


-- Jack Krupansky

-Original Message- 
From: bbarani

Sent: Thursday, October 25, 2012 12:04 PM
To: solr-user@lucene.apache.org
Subject: How to boost exact match?

Hi,

We have a name field which I am boosting using dismax parser.

I want the search to bring the documents that has exact match compared to
partial match.

Ex:

Name:account



Name:account number


Name:account number - closed account


When I search for account number, I want solr to return the document
containing Name:account number first followed by other documents but it
doesnt seem to return that way.. Can someone please help?

Thanks,
Barani


Thanks,
Barani



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-exact-match-tp4015862.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Highlights not returning after upgrading from 3.3 to 4.0

[Daniel Skiles]

I'm running some test on Solr 4.0 before putting it into production, and
I've just encountered an issue with hit highlighting.

I started by placing my index from 3.3 into a Solr 4.0 install.  I then
edited the field definition in the schema config to match my schema from
3.3, with the addition of the new _version_ field.  I also modified the
solrconfig.xml defaults to match my previous 3.3 config (only editing the
default df).

Once I had everything configured, I started the application, then called
"optimize".

After optimization had completed, I executed the following query:

http://
/solr/select?indent=on&version=2.2&q=omega&fq=&start=0&rows=50&fl=*%2Cscore&qt=&wt=&explainOther=&hl=on&hl.fl=contents

This query returned a highlight element in the return packet, but the
fragment section was empty in Solr 4, while it returned fragments in 3.3.
Do I need to make any additional changes?  The default field is "contents",
which is a text_en field.

How to boost exact match?

[bbarani]

Hi,

We have a name field which I am boosting using dismax parser.

I want the search to bring the documents that has exact match compared to
partial match.

Ex:

Name:account



Name:account number


Name:account number - closed account


When I search for account number, I want solr to return the document
containing Name:account number first followed by other documents but it
doesnt seem to return that way.. Can someone please help?

Thanks,
Barani


Thanks,
Barani



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-exact-match-tp4015862.html
Sent from the Solr - User mailing list archive at Nabble.com.

ShardHandler - distribution to non-default request handler doesn't work

[AlexeyK]

I tried to define a suggest component as appears in Wiki.
I also defined a specific /suggest request handler.
This doesn't work in SolrCloud setup, as the query is distributed to the
default /select handler instead.
Specifically, shard handler gets default urls and other cores forward to
/select.
setup:
1 node, 4 cores, 2 shards.

If I try to define a suggest component as a single one for the handler, the
query doesn't get distributed as well.

Configuration:



  suggest
  org.apache.solr.spelling.suggest.Suggester
  org.apache.solr.spelling.suggest.tst.TSTLookup
  
  text  
  0.005
  true


  
  

  true
  suggest
  true
  5
  true


  suggest

  



--
View this message in context: 
http://lucene.472066.n3.nabble.com/ShardHandler-distribution-to-non-default-request-handler-doesn-t-work-tp4015855.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr question regarding handlers and escaping

[Jack Krupansky]

Could you provide a little more detail. I'm not sure if you are simply 
describing behavior you didn't expect or like, or whether there might be an 
actual problem.


-- Jack Krupansky

-Original Message- 
From: sameera somisetty

Sent: Wednesday, October 24, 2012 6:47 PM
To: solr-user@lucene.apache.org
Subject: Solr question regarding handlers and escaping

a difference I found b/w mlt handler and normal query using /select and
mlt=true is that fq(filter query) when applied to the second one is
appended to the main query and not the result. 

Re: UnsupportedOperationException: ExternalFileField (SOLVED)

[Carrie Coy]

The problem seems to have been caused by my failure to completely remove 
the existing index files when I switched the inStock field from an 
indexed boolean field to externally maintained.   After I removed 
everything and re-indexed from scratch, the error went away.


On 10/24/2012 08:57 PM, Carrie Coy wrote:
(Solr4) I'm getting the following error trying to use 
ExternalFileField to maintain an inStock flag.   Any idea what I'm 
doing wrong?


schema.xml:

indexed="false" class="solr.ExternalFileField" valType="float"/>


-rw-r--r-- 1 tomcat tomcat 100434 Oct 24 20:07 external_inStock:
YM0600=1
YM0544=1
YM0505=1

solrconfig.xml:
if(inStock,10,1)


SEVERE: null:java.lang.UnsupportedOperationException
at 
org.apache.solr.schema.ExternalFileField.write(ExternalFileField.java:85)
at 
org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:130)
at 
org.apache.solr.response.JSONWriter.writeSolrDocument(JSONResponseWriter.java:355)
at 
org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:275)
at 
org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:172)
at 
org.apache.solr.response.JSONWriter.writeNamedListAsMapMangled(JSONResponseWriter.java:154)
at 
org.apache.solr.response.PHPWriter.writeNamedList(PHPResponseWriter.java:54)
at 
org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:91)
at 
org.apache.solr.response.PHPResponseWriter.write(PHPResponseWriter.java:36)
at 
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:411)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:289)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at 
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)

at java.lang.Thread.run(Thread.java:662)

Re: one field type extending another

[Jack Krupansky]

Sorry, but not currently. Great suggestion though. I didn't check to see if 
there might already be a Jira for it, but I vaguely recall some discussion 
or mention of it.


-- Jack Krupansky

-Original Message- 
From: Radim Kolar

Sent: Wednesday, October 24, 2012 5:49 PM
To: solr-user@lucene.apache.org
Subject: one field type extending another

can i do something like this: (fails with fieldType: missing mandatory
attribute 'class')
 

one field type will extending another type to save copy and paste. 

Re: SolrCloud leader election on single node

[Mark Miller]

Thanks Alexey -

Could you move this to a JIRA issue? 

- Mark

On Oct 25, 2012, at 7:53 AM, AlexeyK  wrote:

> setup:
> 1 node, 4 cores, 2 shards.
> 15 documents indexed.
> 
> problem:
> init stage times out.
> 
> probable cause:
> According to the init flow, cores are initialized one by one synchronously.
> Actually, the main thread waits
> ShardLeaderElectionContext.waitForReplicasToComeUp until retry threshold,
> while replica cores are *not* yet initialized, in other words there is no
> chance other replicas go up in the meanwhile. 
> stack trace:
> Thread [main] (Suspended) 
>   owns: HashMap  (id=3876)   
>   owns: StandardContext  (id=3877)
>   owns: HashMap  (id=3878)   
>   owns: StandardHost  (id=3879)   
>   owns: StandardEngine  (id=3880) 
>   owns: Service[]  (id=3881)  
>   Thread.sleep(long) line: not available [native method]  
>   ShardLeaderElectionContext.waitForReplicasToComeUp(boolean, String) 
> line:
> 298   
>   ShardLeaderElectionContext.runLeaderProcess(boolean) line: 143  
>   LeaderElector.runIamLeaderProcess(ElectionContext, boolean) line: 152   
>   LeaderElector.checkIfIamLeader(int, ElectionContext, boolean) line: 96  
>   LeaderElector.joinElection(ElectionContext) line: 262   
>   ZkController.joinElection(CoreDescriptor, boolean) line: 733
>   ZkController.register(String, CoreDescriptor, boolean, boolean) line: 
> 566   
>   ZkController.register(String, CoreDescriptor) line: 532 
>   CoreContainer.registerInZk(SolrCore) line: 709  
>   CoreContainer.register(String, SolrCore, boolean) line: 693 
>   CoreContainer.load(String, InputSource) line: 535   
>   CoreContainer.load(String, File) line: 356  
>   CoreContainer$Initializer.initialize() line: 308
>   SolrDispatchFilter.init(FilterConfig) line: 107 
>   ApplicationFilterConfig.getFilter() line: 295   
>   ApplicationFilterConfig.setFilterDef(FilterDef) line: 422   
>   ApplicationFilterConfig.(Context, FilterDef) line: 115
>   StandardContext.filterStart() line: 4072
>   StandardContext.start() line: 4726  
>   StandardHost(ContainerBase).addChildInternal(Container) line: 799   
>   StandardHost(ContainerBase).addChild(Container) line: 779   
>   StandardHost.addChild(Container) line: 601  
>   HostConfig.deployDescriptor(String, File, String) line: 675 
>   HostConfig.deployDescriptors(File, String[]) line: 601  
>   HostConfig.deployApps() line: 502   
>   HostConfig.start() line: 1317   
>   HostConfig.lifecycleEvent(LifecycleEvent) line: 324 
>   LifecycleSupport.fireLifecycleEvent(String, Object) line: 142   
>   StandardHost(ContainerBase).start() line: 1065  
>   StandardHost.start() line: 840  
>   StandardEngine(ContainerBase).start() line: 1057
>   StandardEngine.start() line: 463
>   StandardService.start() line: 525   
>   StandardServer.start() line: 754
>   Catalina.start() line: 595  
>   NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
> available [native method] 
>   NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available   
>   DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not 
> available   
>   Method.invoke(Object, Object...) line: not available
>   Bootstrap.start() line: 289 
>   Bootstrap.main(String[]) line: 414
> 
>   
> After a while, the session times out and following exception appears:
> Oct 25, 2012 1:16:56 PM org.apache.solr.cloud.ShardLeaderElectionContext
> waitForReplicasToComeUp
> INFO: Waiting until we see more replicas up: total=2 found=0 timeoutin=-95
> Oct 25, 2012 1:16:56 PM org.apache.solr.cloud.ShardLeaderElectionContext
> waitForReplicasToComeUp
> INFO: Was waiting for replicas to come up, but they are taking too long -
> assuming they won't come back till later
> Oct 25, 2012 1:16:56 PM org.apache.solr.common.SolrException log
> SEVERE: Errir checking for the number of election
> participants:org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for
> /collections/collection1/leader_elect/shard2/election
>   at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
>   at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>   at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
>   at
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:227)
>   at
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:224)
>   at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:63)
>   at
> org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:224)
>   at
> org.apache.solr.cloud.ShardLeaderElectionContext.waitF

Re: org.apache.lucene.queryparser.classic.ParseException - a Bug?

[Ahmet Arslan]

Hi Deniz,

Haven't used geofilt query parser but with solrJ you don't need URL encoding. 
Use solrQuery.set("fq","{!geofilt sfield=store}"); instead.



--- On Thu, 10/25/12, deniz  wrote:

> From: deniz 
> Subject: org.apache.lucene.queryparser.classic.ParseException - a Bug?
> To: solr-user@lucene.apache.org
> Date: Thursday, October 25, 2012, 7:18 AM
> Hi all,
> 
> I was trying to provide spatial search via solrj client. but
> when i try to
> run it i got 
> 
> 
> org.apache.solr.common.SolrException:
> org.apache.lucene.queryparser.classic.ParseException:
> Expected identifier at
> pos 9 str='{!geofilt+sfield=store}'
> 
> 
> I have tried to do the same search on browser and via URL
> request from java
> and there was no problem with those... but via solrj i keep
> getting the
> error above... 
> 
> below is the code that I have used to reproduce the error 
> 
> 
> 
> import java.io.BufferedReader;
> import java.io.InputStreamReader;
> import java.io.OutputStreamWriter;
> import java.net.URL;
> import java.net.URLConnection;
> 
> import org.apache.solr.client.solrj.SolrQuery;
> import org.apache.solr.client.solrj.impl.HttpSolrServer;
> import org.apache.solr.client.solrj.response.QueryResponse;
> import org.apache.solr.common.SolrDocumentList;
> 
> 
> public class RunMeFast {
>     
>     
>  public static void main(String[] args)
>  {
>  try{
>  HttpSolrServer server = new
> HttpSolrServer("http://localhost:8983/solr/testcore";);
>  SolrQuery solrQuery = new
> SolrQuery();
>     
>     
>  solrQuery.set("q", "*:*");
>  solrQuery.set("d","1000");
>  solrQuery.set("fl","*,_dist_:geodist()");
>  solrQuery.set("sfield","store");
>  solrQuery.set("pt","47,+8");
>  solrQuery.set("fq","{!geofilt+sfield=store}");
> 
>  QueryResponse   response
> = server.query(solrQuery);
>  long totalDocs =
> response.getResults().getNumFound();
>  SolrDocumentList docList =
> response.getResults();
>  System.out.println(docList.toString());
>  }
>  catch(Exception e ){
>    
>  e.printStackTrace();
>  }
>  
>  
>  try
>  {
>      URL url =
> new
> URL("http://localhost:8983/solr/testcore/select?q=*:*&d=1000&fl=*,_dist_:geodist()&sfield=store&pt=47,+8&fq={!geofilt+sfield=store}");
>    
>  URLConnection conn =
> url.openConnection();
>            
> conn.setDoOutput(true);
>            
> OutputStreamWriter wr = new
> OutputStreamWriter(conn.getOutputStream());
>            
> wr.flush();
> 
>             // Get
> the response
>            
> BufferedReader rd = new BufferedReader(new
> InputStreamReader(conn.getInputStream()));
>             String
> line;
>             while
> ((line = rd.readLine()) != null) {
>              
>   System.out.println(line);   
>         }
>            
> wr.close();
>            
> rd.close();
>      
>  }catch(Exception e){
>    
>  e.printStackTrace();
>  }
>  
>  }
>     
> 
> 
> 
> And the output is :
> 
> 
> SOLRJ Call:
> 
> Oct 25, 2012 11:54:24 AM
> org.apache.solr.client.solrj.impl.HttpClientUtil
> createClient
> INFO: Creating new http client,
> config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> org.apache.solr.common.SolrException:
> org.apache.lucene.queryparser.classic.ParseException:
> Expected identifier at
> pos 9 str='{!geofilt+sfield=store}'
>     at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:401)
>     at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
>     at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
>     at
> org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
>     at RunMeFast.main(RunMeFast.java:29)
> 
> URL Request:
> 
> 
> correct doc list
> 
> 
> 
> 
> I am using Solr 4.0...
> 
> so is this a bug? or simply a mistake? anyone can help me?
> 
> 
> 
> 
> 
> 
> 
> 
> -
> Zeki ama calismiyor... Calissa yapar...
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/org-apache-lucene-queryparser-classic-ParseException-a-Bug-tp4015763.html
> Sent from the Solr - User mailing list archive at
> Nabble.com.
>

Re: Improving performance for use-case where large (200) number of phrase queries are used?

[Peter Keegan]

Yes #5 is the same thing (sorry, I didn't read them all thoroughly). Your
description of the phrases being 'tags' suggests that you don't need term
positions for matching, and as you noted, you would get unwanted partial
matches. And, the TermQuerys would be much faster.

Peter


On Wed, Oct 24, 2012 at 8:33 PM, Aaron Daubman  wrote:

> Hi Peter,
>
> Thanks for the recommendation - I believe we are thinking along the
> same lines, but wanted to check to make sure. Are you suggesting
> something different than my #5 (below) or are we essentially
> suggesting the same thing?
>
> On Wed, Oct 24, 2012 at 1:20 PM, Peter Keegan 
> wrote:
> > Could you index your 'phrase tags' as single tokens? Then your phrase
> > queries become simple TermQuerys.
>
> >>
> >> 5) *This is my current favorite*: stop tokenizing/analyzing these
> >> terms and just use KeywordTokenizer. Most of these phrases are
> >> pre-vetted, and it may be possible to clean/process any others before
> >> creating the docs. My main worry here is that, currently, if I
> >> understand correctly, a document with the phrase "brazilian pop" would
> >> still be returned as a match to a seed document containing only the
> >> phrase "brazilian" (not the other way around, but that is not
> >> necessary), however, with KeywordTokenizer, this would no longer be
> >> the case. If I switched from the current dubious tokenize/stem/etc...
> >> and just used Keyword, would this allow queries like "this used to be
> >> a long phrase query" to match documents that have "this used to be a
> >> long phrase query" as one of the multivalued values in the field
> >> without having to pull term positions? (and thus significantly speed
> >> up performance).
> >>
>
> Thanks again,
>  Aaron
>

Re: MMapDirectory, demand paging, lazy evaluation, ramfs and the much maligned RAMDirectory (oh my!)

[Erick Erickson]

You may well have already seen this, but in case not:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

FWIW,
Erick

On Wed, Oct 24, 2012 at 9:51 PM, Shawn Heisey  wrote:
> On 10/24/2012 6:29 PM, Aaron Daubman wrote:
>>
>> Let me be clear that that I am not interested in RAMDirectory.
>> However, I would like to better understand the oft-recommended and
>> currently-default MMapDirectory, and what the tradeoffs would be, when
>> using a 64-bit linux server dedicated to this single solr instance,
>> with plenty (more than 2x index size) of RAM, of storing the index
>> files on SSDs versus on a ramfs mount.
>>
>> I understand that using the default MMapDirectory will allow caching
>> of the index in-memory, however, my understanding is that mmaped files
>> are demand-paged (lazy evaluated), meaning that only after a block is
>> read from disk will it be paged into memory - is this correct? is it
>> actually block-by-block (page size by page size?) - any pointers to
>> decent documentation on this regardless of the effectiveness of the
>> approach would be appreciated...
>
>
> You are correct that the data must have just been accessed to be in the disk
> cache.This does however include writes -- so any data that gets indexed will
> be in the cache because it has just been written.  I do believe that it is
> read in one page block at a time, and I believe that the blocks are 4k in
> size.
>
>
>> My concern with using MMapDirectory for an index stored on disk (even
>> SSDs), if my understanding is correct, is that there is still a large
>> startup cost to MMapDirectory, as it may take many queries before even
>> most of a 20G index has been loaded into memory, and there may yet
>> still be "dark corners" that only come up in edge-case queries that
>> cause QTime spikes should these queries ever occur.
>>
>> I would like to ensure that, at startup, no query will incur
>> disk-seek/read penalties.
>>
>> Is the "right" way to achieve this to copy the index to a ramfs (NOT
>> ramdisk) mount and then continue to use MMapDirectory in Solr to read
>> the index? I am under the impression that when using ramfs (rather
>> than ramdisk, for which this would not work) a file mmaped on a ramfs
>> mount will actually share the same address space, and so would not
>> incur the typical double-ram overhead of mmaping a file in memory just
>> o have yet another copy of the file created in a second memory
>> location. Is this correct? If not, would you please point me to
>> documentation stating otherwise (I haven't found much documentation
>> either way).
>
>
> I am not familiar with any "double-ram overhead" from using mmap.  It should
> be extroardinarily efficient, so much so that even when your index won't fit
> in RAM, performance is typically still excellent.  Using an SSD instead of a
> spinning disk will increase performance across the board, until enough of
> the index is cached in RAM, after which it won't make a lot of difference.
>
> My parting thoughts, with a general note to the masses: Do not try this if
> you are not absolutely sure your index will fit in memory!  It will tend to
> cause WAY more problems than it will solve for most people with large
> indexes.
>
> If you actually do have considerably more RAM than your index size, and you
> know that the index will never grow to where it might not fit, you can use a
> simple trick to get it all cached, even before running queries.  Just read
> the entire contents of the index, discarding everything you read.  There are
> two main OS variants to consider here, and both can be scripted, as noted
> below.  Run the command twice to see the difference that caching makes for
> the second run.  Note that an SSD would speed the first run of these
> commands up considerably:
>
> *NIX (may work on a mac too):
> cat /path/to/index/files/* > /dev/null
>
> Windows:
> type C:\Path\To\Index\Files\* > NUL
>
> Thanks,
> Shawn
>

Facet date/range + facet.mincount + distributed search issue

[Dovao Jimenez, Oscar]

Dear Mrs/Mr,

Using facet date/range on a date typed field and on a distributed search 
between schema compatible cores, the use of facet.mincount=1 brings a cut down 
number of facet values (over 500 facet values expected, 5 facet values 
retrieved). I wonder whether facet.mincount is supported on distributed 
searches as it seems to work well on single core ones. I'm using Solr 4.0 full 
release by the way.

Please, do let me know if you need any other detail about this issue.

Many thanks in advance,

Oscar Dovao Jimenez
Java Applications Developer

SolrCloud leader election on single node

[AlexeyK]

setup:
1 node, 4 cores, 2 shards.
15 documents indexed.

problem:
init stage times out.

probable cause:
According to the init flow, cores are initialized one by one synchronously.
Actually, the main thread waits
ShardLeaderElectionContext.waitForReplicasToComeUp until retry threshold,
while replica cores are *not* yet initialized, in other words there is no
chance other replicas go up in the meanwhile. 
stack trace:
Thread [main] (Suspended)   
owns: HashMap  (id=3876)   
owns: StandardContext  (id=3877)
owns: HashMap  (id=3878)   
owns: StandardHost  (id=3879)   
owns: StandardEngine  (id=3880) 
owns: Service[]  (id=3881)  
Thread.sleep(long) line: not available [native method]  
ShardLeaderElectionContext.waitForReplicasToComeUp(boolean, String) 
line:
298 
ShardLeaderElectionContext.runLeaderProcess(boolean) line: 143  
LeaderElector.runIamLeaderProcess(ElectionContext, boolean) line: 152   
LeaderElector.checkIfIamLeader(int, ElectionContext, boolean) line: 96  
LeaderElector.joinElection(ElectionContext) line: 262   
ZkController.joinElection(CoreDescriptor, boolean) line: 733
ZkController.register(String, CoreDescriptor, boolean, boolean) line: 
566   
ZkController.register(String, CoreDescriptor) line: 532 
CoreContainer.registerInZk(SolrCore) line: 709  
CoreContainer.register(String, SolrCore, boolean) line: 693 
CoreContainer.load(String, InputSource) line: 535   
CoreContainer.load(String, File) line: 356  
CoreContainer$Initializer.initialize() line: 308
SolrDispatchFilter.init(FilterConfig) line: 107 
ApplicationFilterConfig.getFilter() line: 295   
ApplicationFilterConfig.setFilterDef(FilterDef) line: 422   
ApplicationFilterConfig.(Context, FilterDef) line: 115
StandardContext.filterStart() line: 4072
StandardContext.start() line: 4726  
StandardHost(ContainerBase).addChildInternal(Container) line: 799   
StandardHost(ContainerBase).addChild(Container) line: 779   
StandardHost.addChild(Container) line: 601  
HostConfig.deployDescriptor(String, File, String) line: 675 
HostConfig.deployDescriptors(File, String[]) line: 601  
HostConfig.deployApps() line: 502   
HostConfig.start() line: 1317   
HostConfig.lifecycleEvent(LifecycleEvent) line: 324 
LifecycleSupport.fireLifecycleEvent(String, Object) line: 142   
StandardHost(ContainerBase).start() line: 1065  
StandardHost.start() line: 840  
StandardEngine(ContainerBase).start() line: 1057
StandardEngine.start() line: 463
StandardService.start() line: 525   
StandardServer.start() line: 754
Catalina.start() line: 595  
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]   
NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available   
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not 
available   
Method.invoke(Object, Object...) line: not available
Bootstrap.start() line: 289 
Bootstrap.main(String[]) line: 414


After a while, the session times out and following exception appears:
Oct 25, 2012 1:16:56 PM org.apache.solr.cloud.ShardLeaderElectionContext
waitForReplicasToComeUp
INFO: Waiting until we see more replicas up: total=2 found=0 timeoutin=-95
Oct 25, 2012 1:16:56 PM org.apache.solr.cloud.ShardLeaderElectionContext
waitForReplicasToComeUp
INFO: Was waiting for replicas to come up, but they are taking too long -
assuming they won't come back till later
Oct 25, 2012 1:16:56 PM org.apache.solr.common.SolrException log
SEVERE: Errir checking for the number of election
participants:org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/collections/collection1/leader_elect/shard2/election
at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
at
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:227)
at
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:224)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:63)
at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:224)
at
org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:276)
at
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:143)
at
org.apache.solr.cloud.L

Query elevation component fails

[AlexeyK]

Using SolrCloud release with following configuration:



string
elevate.xml
  

  
  

  explicit
  text


  elevator

  

Running the query
http://localhost:8080/solr/collection1/elevate?q=evelatedtext

constantly getting the following exception:

SEVERE: null:java.lang.IndexOutOfBoundsException: Index: 1, Size: 0
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.apache.solr.common.util.NamedList.getVal(NamedList.java:136)
at
org.apache.solr.handler.component.ShardFieldSortedHitQueue$ShardComparator.sortVal(ShardDoc.java:217)
at
org.apache.solr.handler.component.ShardFieldSortedHitQueue$2.compare(ShardDoc.java:255)
at
org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardDoc.java:159)
at
org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardDoc.java:101)
at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:231)
at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
at
org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
at
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:863)
at
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626)
at
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:309)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at 
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)


The lookup is made for the "_elevate_" sort field. Should I have such a
field in schema?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Query-elevation-component-fails-tp4015793.html
Sent from the Solr - User mailing list archive at Nabble.com.

one field type extending another

[Radim Kolar]

can i do something like this: (fails with fieldType: missing mandatory 
attribute 'class')
 termVectors="true" termPositions="true" termOffsets="true"/>


one field type will extending another type to save copy and paste.