Re: Performance potential for updating (reindexing) documents

2016-04-01 Thread Erick Erickson
Shawn: bq: The bottleneck is definitely Solr. Since you commented out the server.add(doclist), you're right to focus there. I've seen a few things that help. 1> batch the documents, i.e. in the doclist above the list should be on the order of 1,000 docs. Here are some numbers I worked up one

Re: make document with more matches rank higher with edismax parser?

2016-04-01 Thread Alexandre Rafalovitch
Have you tried 'tie' parameter? https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser#TheDisMaxQueryParser-Thetie%28TieBreaker%29Parameter Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On 1 April 2016 at

Re: Search over XML data using xpath

2016-04-01 Thread Alexandre Rafalovitch
You may be interested in checking out: http://luxdb.org/ - seems to be still on Solr 4.x though http://siren.solutions/siren/overview/ - I believe they indexed XML Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On 1 April 2016 at

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Girish Tavag
Hi Shawn and Jack, Yes that is true. I was referring the tutorial and practicing and ended up in this. By the way, one more thing I would like to know, is it possible to schedule the full data import? or the delta import? i.e. on regular intervals the data should be updated. Regards, GNT On

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Shawn Heisey
On 4/1/2016 1:24 PM, Girish Tavag wrote: > Finally i'm able to figure out the problem. The issue was in > solrconfig.xml where the booleans was defined. I replaced booleans with > boolean and other similar fileds and it worked correctly :) This has happened because you mixed the solrconfig.xml

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Jack Krupansky
I think it's a bug... Ah, the key clue is here: Caused by: org.apache.solr.common.SolrException: fieldType 'booleans' not found in the schema at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessor Factory$TypeMapping.populateValueClasses(AddSchemaFieldsUpdateProcessor

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Girish Tavag
Hi Shawn, Finally i'm able to figure out the problem. The issue was in solrconfig.xml where the booleans was defined. I replaced booleans with boolean and other similar fileds and it worked correctly :) Regards, GNT.

Re: Sorting question

2016-04-01 Thread John Bickerstaff
Tamas, This feels a bit like a "user favorites" problem. I did a little searching and found this... Don't know if it will help, but when I'm looking for stuff like this I find it helps to try to come up with generic or different descriptions of my problem and go search those as well...

Re: Sorting question

2016-04-01 Thread John Bickerstaff
Tamas, I'm brainstorming here - not being careful, just throwing out ideas... One thing that comes up is a separate document in SOLR - one doc for each list. If a user adds a doc to their list, that doc's id gets added to this other type of document... So, a document with the title "List 1"

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Girish Tavag
Hi Shawn, Thank you for responding and informing me about the attachments. Here is the log file details.. 016-04-01 18:24:08.191 INFO (coreLoadExecutor-6-thread-1) [ x:myDatabase] o.a.s.c.CachingDirectoryFactory looking to close solr-5.5.0\server\solr\myDatabase\data\index

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Shawn Heisey
On 4/1/2016 12:11 PM, Girish Tavag wrote: > I copied schema.xml from solr-5.5.0\example\example-DIH\solr\db\conf\ > to \solr-5.5.0\server\solr\myDatabase\conf\ > I've attached the file too. > > @Shawn > The file does not have any field which defined as name="somefield" type="booleans"

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Girish Tavag
Here is the error message "*myDatabase:* org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: fieldType 'booleans' not found in the schema" On Fri, Apr 1, 2016 at 11:41 PM, Girish Tavag wrote: > Hi Jack, > > I copied schema.xml from

Re: Slor 5.5.0 : SolrException: fieldType 'booleans' not found in the schema

2016-04-01 Thread Girish Tavag
Hi Jack, I copied schema.xml from solr-5.5.0\example\example-DIH\solr\db\conf\ to \solr-5.5.0\server\solr\myDatabase\conf\ I've attached the file too. @Shawn The file does not have any field which defined as On Fri, Apr 1, 2016 at 9:13 AM, Jack Krupansky wrote:

Re: Sorting question

2016-04-01 Thread John Bickerstaff
OK - I get it. List order is totally arbitrary and cannot be tied to an hard data point. I'll have to think - Perhaps billnbell's solution will help, although I'm not totally sure I understand that suggestion yet. At this point, you could get all the documents for List X that match the search

Re: Sorting question

2016-04-01 Thread billnbell
Put the match into 2 separate fields and index it. Then sort in Solr by the 2 fields is one way Bill Bell Sent from mobile > On Apr 1, 2016, at 11:15 AM, John Bickerstaff > wrote: > > Just to be clear - I don't mean who requests the list (application or user) > I

Re: Sorting question

2016-04-01 Thread Tamás Barta
So, the list order is determined by the user. The user creates a list, adds products to it and i have to display these list using filters and pagination. Let's assume there is list with 1 products in it. In the website where i display the list only 50 products are displayed in a page. So if i

Re: Sorting question

2016-04-01 Thread John Bickerstaff
Oh - and if you send a copy of your query - please include a human-readable version of what your intent is... Something like: Find all the documents that have "blue" in the color field in addition to searching the title field for the user's search term..." ...Or whatever your intent is for this

Re: Sorting question

2016-04-01 Thread John Bickerstaff
Just to be clear - I don't mean who requests the list (application or user) I mean what "rule" determines the ordering of the list? Or, is there even a rule of any kind? In other words, does a user arbitrarily decide that documentA, documentF, and documentW should be on a list of their own? For

Re: Sorting question

2016-04-01 Thread Tamás Barta
Some of the lists are created by users and some are generated by applications, it doesn't matter. It would be fine to solve it in Solr because Solr does the work of filtering and pagination. If sorting were done outside than I would have to read every document from Solr to sort them. It is not an

Re: Sorting question

2016-04-01 Thread John Bickerstaff
Specifically, what drives the position in the list? Is it arbitrary or is it driven by some piece of data? If data-driven - code could do the sorting based on that data... separate from SOLR... Alternatively, if the data point exists in SOLR, a "sub-query" might be used to get the right sort

Function Query Parsing problem in Solr 5.4.1 and Solr 5.5.0

2016-04-01 Thread Max Bridgewater
Hi, I have the following configuration for firstSearcher handler in solrconfig.xml: parts score desc, Review1 asc, Rank2 asc make {!func}sum(product(0.01,param1), product(0.20,param2), min(param2,0.4)) desc

Re: Sorting question

2016-04-01 Thread Tamás Barta
For example I have to display sellable products which are in list X in the correct order. If I add a "status" and "list" (multivalued) fields to every document (products), then I can execute a query: status:sellable AND list:X, where X is the ID of the list. The list field contains IDs of the

Re: Sorting question

2016-04-01 Thread Alessandro Benedetti
I think this is a classic XY Problem , you are trying to solve X with Y , and you are asking us about Y . Could you describe us what is your X problem ? What are you trying to do with this ordered lists ? If not I would add a field to the product called : list_position ( or a similar name) of

Re: Sorting question

2016-04-01 Thread Binoy Dalal
I don't think I understand your problem properly. Are you trying to pre-sort the products? On Fri, 1 Apr 2016, 19:49 Tamás Barta, wrote: > Hi, > > I have a problem and I don't know how should I solve it in Solr. > > I have products indexed. Every product can be in lists.

Sorting question

2016-04-01 Thread Tamás Barta
Hi, I have a problem and I don't know how should I solve it in Solr. I have products indexed. Every product can be in lists. It is possible that a product isn't in any list or it is in multiple list. In a list the products are ordered. I would like to search for products in a specified list with

Re: Search over XML data using xpath

2016-04-01 Thread Alessandro Benedetti
Let's try to make clarity in here : Lucene Query Syntax is not XPath. This means you can not search Lucene Documents as you do for xml nodes. You need to model your information according to the Lucene Document ( and children) structure. Then you can play with the Lucene Query language and the

Re: Deleted documents and expungeDeletes

2016-04-01 Thread David Santamauro
The docs on reclaimDeletesWeight say: "Controls how aggressively merges that reclaim more deletions are favored. Higher values favor selecting merges that reclaim deletions." I can't imagine you would notice anything after only a few commits. I have many shards that size or larger and what

Re: Deleted documents and expungeDeletes

2016-04-01 Thread Jostein Elvaker Haande
On 30 March 2016 at 17:46, Erick Erickson wrote: > through a clever bit of reflection, you can set the > reclaimDeletesWeight variable from solrconfig by including something > like > 5 (going from memory > here, you'll get an error on startup if I've messed it up.) I

Search over XML data using xpath

2016-04-01 Thread Miguel Valencia Zurera
Hi everybody I'm looking for the way to store XML file and keep on hierarchy of the data because I need show full xml and besides to search inside of nodes of xml. Only I have found XPathEntityProcessor for import xml but it does not keep on the hierarchy of the data. I have not found one

How to create mappings between multiple Solr docs

2016-04-01 Thread vivekaltruist
I am using Solr 5.4 and have Solr schema containing 5 fields. Now, I am indexing two types of docs: Doc1: field1 field2 field3 Doc2: field1 field4 field5 Here field1 contains unique ID and it is common to both the docs. There are more docs of type Doc2 than Doc1. Since