boosting

2012-01-11 Thread Finotti Simone
Hello ML, I wonder if it is possibile to define a boost for certains fields in schema.xml configuration. As far, I have found ways to define a boost while indexing and while querying, so I suspect the straight answer is no. Anyway, I'd like a confirm, if possible. Thank you in advance S

Highlighting issue with PlainTextEntityProcessor.

2012-01-11 Thread meghana
Hi all, My Solr Configuration had one multi-valued field which is imported using XPathEntityProcess and TemplateTransformer . Then we had to convert it to non-multivlaued field, We did that using PlainTextEntityProcessor and Script Transformer. Search on my old configuration was working fine ,

Search Issue

2012-01-11 Thread Rohit
Hi, We are storing a large number of tweets and blogs feeds into solr. Now if the user searches for twitter mentions like, @rohit , records which just contain the word rohit are also being returned. Even if we do an exact match @rohit, I understand this happens because of use of

Re: Multiple Sort for Group/Folding

2012-01-11 Thread Martijn v Groningen
Hi Mauro, During the first pass search the sort param is used to determine the top N groups. Then during the second pass search the documents inside the top N groups are sorted using the group.sort parameter. The group.sort doesn't change how the groups them self are sorted. Martijn On 11

Not able to see the output when search is clicked

2012-01-11 Thread Rajdeep Alapati
Hi, In my SOLR, I have a query based data-config written and was able to manage below steps but i was not able to see the output 1) Register Data Import Request handler in Solr-config.xml 2) Modify Data-Config.xml for the appropriate query to get data imported from which includes making use

SpatialSearch, geofilt and documents missing a value in sfield

2012-01-11 Thread Tanguy Moal
Dear ML, I'm performing some developments relying on spatial capabilities of solr. I'm using Solr 3.5, have been reading http://wiki.apache.org/solr/SpatialSearch#Spatial_Query_Parameters and have the basic behaviours I wanted working. I use geofilt on a latlong field, with geodist() in the

Large data set or data corpus

2012-01-11 Thread findbestopensource
Hello all, Recently i saw couple of discussions in LinkedIn group about generating large data set or data corpus. I have compiled the same in to an article. Hope it would be helpful. If you have any other links where we could get large data set for free, please reply to this mail thread, i will

Re: Search Issue

2012-01-11 Thread findbestopensource
While indexing @ is removed. You need to use your own Tokenizer which will consider @rohit as one word. Another option is to break the tweet in to two fields, @username and the tweet. Index both the fields but don't use any tokenizer for the field @username. Just index as it is. While querying

solrcloud replicating new cores

2012-01-11 Thread Phil Hoy
Hi, Is it possible to configure solr using solrcloud and the distribution handler such that if a new core is added to the master then that core is added and replicated to the slaves. Phil

Re: Not able to see the output when search is clicked

2012-01-11 Thread Gora Mohanty
On Wed, Jan 11, 2012 at 4:31 PM, Rajdeep Alapati rajdeep.alap...@benefitfocus.com wrote: [...] 5) Make full import http request for data to get indexed into solr server. Even though i see that all the rows are indexed but not able to find results when search is clicked on the admin page

RE: Not able to see the output when search is clicked

2012-01-11 Thread Rajdeep Alapati
Hi Mohanty, I have entered *:* in the search box and the url it redirected to was http://localhost:8983/solr/select/?q=*%3A*version=2.2start=0rows=10indent=on; and the output was the below content where i expected 3 rows in 3 items Thanks Raj Deep Alapati ?xml version=1.0

exception while loading with DIH multi-threaded

2012-01-11 Thread Bernd Fehling
Hi list, after changing DIH to multi-theaded (4 threads) I get sometimes an exception. This is not always the case and I never had any problems with single-threaded at all. I'm using Solr 3.5 but also tried branch_3x (3.6) and could see this with both versions. Don't know why this comes up

Call for Submission Berlin Buzzwords 2012all for Submission Berlin Buzzwords - http://berlinbuzzwords.de

2012-01-11 Thread Simon Willnauer
Call for Submission Berlin Buzzwords 2012 - Search, Store, Scale  -- June 4 / 5. 2012 The event will comprise presentations on scalable data processing. We invite you to submit talks on the topics:  * IR / Search - Lucene, Solr, katta, ElasticSearch or comparable solutions  * NoSQL - like

Re: Getting started with indexing a database

2012-01-11 Thread Erick Erickson
I'm not going to be much help here since DIH is a mystery to me, I usually go with a SolrJ program when DIH gets beyond simple cases. But have you seen: http://wiki.apache.org/solr/DataImportHandler#interactive It's a tool that helps you see what's going on with your query. Best Erick On Mon,

Relevancy and random sorting

2012-01-11 Thread Alexandre Rocco
Hello all, Recently i've been trying to tweak some aspects of relevancy in one listing project. I need to give a higher score to newer documents and also boost the document based on a boolean field that indicates the listing has pictures. On top of that, in some situations we need a random

Re: exception while loading with DIH multi-threaded

2012-01-11 Thread Bernd Fehling
After browsing through the issues it looks like something belonging to https://issues.apache.org/jira/browse/SOLR-2694 Am 11.01.2012 14:08, schrieb Bernd Fehling: Hi list, after changing DIH to multi-theaded (4 threads) I get sometimes an exception. This is not always the case and I never

Re: Finding single word index data based on multiple word query

2012-01-11 Thread Erick Erickson
Dave: That's actually an interesting way to use WordDelimiterFitlerFactory. I think you're being bitten by the difference between analysis and query parsing. The analysis page bypasses query parsing and throws the input against the exact field you specified, without any, well, parsing. But when

Re: Question about updating index with custom field types

2012-01-11 Thread Erick Erickson
I'm not sure what custom field types have to do with XML here. Somewhere, you have to have defined a *field* in your schema.xml that references your custom type, something like: field name=the_offer type=offer . / then the XML is just like any other field doc field name=the_offer

Re: Solr core as a dispatcher

2012-01-11 Thread Hector Castro
In our setup, we handle the document distribution and uniqueness across cores outside of Solr. -- Hector On Jan 11, 2012, at 1:53 AM, shlomi java wrote: Straying a bit from the subject, don't you think it will be useful to have the shards parameter used also in the index, in order to

catchall field minus one field

2012-01-11 Thread elisabeth benoit
Hello, I have a catchall field, and I need to do some request in all fields of that catchall field, minus one. To avoid duplicating my index, I'd like to know if there is a way to use my catch field while excluding that one field. Thanks, Elisabeth

Re: Not able to see the output when search is clicked

2012-01-11 Thread Erick Erickson
Several questions: 1 did you commit after the full import? 2 have you tried looking at the interactive DIH console? See: http://wiki.apache.org/solr/DataImportHandler#interactive 3 Do your Solr logs show anything? Best Erick On Wed, Jan 11, 2012 at 7:27 AM, Rajdeep Alapati

Re: boosting

2012-01-11 Thread Erick Erickson
I don't know of any way to do this. The closest you can come to what I infer you want would be to define an edismax query handler in solrconfig.xml and assign boosts automatically when querying. Or you can apply boosts at index time by specifying a boost attribute in the XML. Best Erick On Wed,

Re: Not able to see the output when search is clicked

2012-01-11 Thread Gora Mohanty
On Wed, Jan 11, 2012 at 5:57 PM, Rajdeep Alapati rajdeep.alap...@benefitfocus.com wrote: Hi Mohanty, I have entered *:* in the search box and the url it redirected to was http://localhost:8983/solr/select/?q=*%3A*version=2.2start=0rows=10indent=on; OK, this looks fine: I was just wondering

Re: Getting started with indexing a database

2012-01-11 Thread Gora Mohanty
On Tue, Jan 10, 2012 at 7:09 AM, Mike O'Leary tmole...@uw.edu wrote: [...] My data-config.xml file looks like this: dataConfig  dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver              url=jdbc:mysql://localhost:3306/bioscope user=db_user password=/  document name=bioscope

Re: Highlighting issue with PlainTextEntityProcessor.

2012-01-11 Thread Erick Erickson
Well, are you sure that the data is *in* the index? Did you change maxFieldLength in solrconfig.xml? First, I'd be sure that the data is in the stored field to highlight. Best Erick Out of curiosity, why did you have to change from multValued? On Wed, Jan 11, 2012 at 3:29 AM, meghana

Re: solrcloud replicating new cores

2012-01-11 Thread Mark Miller
On Wed, Jan 11, 2012 at 6:38 AM, Phil Hoy p...@friendsreunited.co.ukwrote: Hi, Is it possible to configure solr using solrcloud and the distribution handler such that if a new core is added to the master then that core is added and replicated to the slaves. Phil Hey Phil, Not currently.

Re: Relevancy and random sorting

2012-01-11 Thread Erick Erickson
I really don't understand what this means: random sorting for the records but also preserving the ranking Either you're sorting on rank or you're not. If you mean you're trying to shift things around just a little bit, *mostly* respecting relevance then I guess you can do what you're thinking.

Re: catchall field minus one field

2012-01-11 Thread Erick Erickson
Hmmm, Once the data is included in the catch-all, it's indistinguishable from all the rest of the data, so I don't see how you could do this. A clause like: -excludeField:[* TO *] would exclude all documents that had any data in the field so that's probably not what you want. Could you

Re: Solr core as a dispatcher

2012-01-11 Thread shlomi java
My point is this - *everyone *using shards needs to handle that uniqueness. Why not make it as a mechanism in Solr, with the ability to plugin our own implementation of the hashing method? Or is that what Solr Cloud is all about? SJ On Wed, Jan 11, 2012 at 4:33 PM, Hector Castro

Re: Multiple Sort for Group/Folding

2012-01-11 Thread Mauro Asprea
Thanks Martijn. I know what you mean, but in my case I want to sort the groups using both sort keys and then inside each group it should use the group.sort. If you take a look at the query you will see both regular sorting keys sort=location_weight_i desc, weight_i desc. The issue here is

Upgrading from 1.4 to the latest version

2012-01-11 Thread ahammad
I was doing some reading on the new features and whatnot, and I am interested in upgrading. I have a few questions though: 1) The index seemed to have changed, can I reuse the current index or should I reindex the data? I read some things about optimizing the index and whatnot, but I am not clear

query parser attaches text: prefix to query

2012-01-11 Thread Matthias Müller
Hi, when I request a query to solr with /solr/select?q=querydebugQuery=trueqf=title I get no result. The debug information tells me this. str name=querystringquery/str str name=parsedquerytext:query/str When I request a query to solr with /solr/select?q=title:querydebugQuery=true I get the

Re: query parser attaches text: prefix to query

2012-01-11 Thread Erik Hatcher
Because you're using the lucene query parser which doesn't use qf. Add defType=dismax to get the effect you want. Erik On Jan 11, 2012, at 12:22, Matthias Müller mm4...@googlemail.com wrote: Hi, when I request a query to solr with /solr/select?q=querydebugQuery=trueqf=title I get no

Re: Relevancy and random sorting

2012-01-11 Thread Alexandre Rocco
Erick, Probably I really written something silly. You are right on either sorting by field or ranking. I just need to change the ranking to shift things around as you said. To clarify the use case: We have a listing aggregator that gets product listings from a lot of different sites and since

Re: exception while loading with DIH multi-threaded

2012-01-11 Thread Mikhail Khludnev
FYI, it's https://issues.apache.org/jira/browse/SOLR-2804 I'm trying to address it. On Wed, Jan 11, 2012 at 5:49 PM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: After browsing through the issues it looks like something belonging to

Fuzzy query spec in edismax

2012-01-11 Thread Walter Underwood
It's my fault for not contributing it back while I was at Netflix, but I need the feature I implemented four years ago to use fuzzy search in the qf parameter. It looks like another person has implemented this as SOLR-629. How do we get this back on the release train? wunder -- Walter

Re: Solr core as a dispatcher

2012-01-11 Thread Ken Krugler
Hi Hector, On Jan 9, 2012, at 4:15pm, Hector Castro wrote: Hi, Has anyone had success with multicore single node Solr configurations that have one core acting solely as a dispatcher for the other cores? For example, say you had 4 populated Solr cores – configure a 5th to be the

Search Specific Boosting

2012-01-11 Thread Brett
I'm implementing a feature where admins have the ability to control the order of the results by adding a boost to any specific search. The search is a faceted interface (no text input) and which we take a hash of the search parameters (to form a unique search id) and then boost that field for

Re: Relevancy and random sorting

2012-01-11 Thread Erick Erickson
Alexandre: Have you thought about grouping? If you can analyze the incoming documents and include a field such that similar documents map to the same value, than group on that value you'll get output that isn't dominated by repeated copies of the similar documents. It depends, though, on being

Re: Upgrading from 1.4 to the latest version

2012-01-11 Thread Erick Erickson
Short form: It's always best to re-index if possible. 1 Should work, indexes are supposed to be backwards-compatible across one release. But I'd still re-index if possible. WARNING: as you add documents etc. when, say, a merge happens, the newly merged segments will NOT be readable

Re: Search Specific Boosting

2012-01-11 Thread Erick Erickson
Boosts are fairly coarse-grained. I suspect your boost factors are just being rounded into the same buckets. Attaching debugQuery=on and looking at how the scores were calculated should help you figure out if this is the case. Best Erick On Wed, Jan 11, 2012 at 7:57 PM, Brett br...@chopshop.org

Re: Relevancy and random sorting

2012-01-11 Thread Ted Dunning
I think the OP meant to use random order in the case of score ties. On Wed, Jan 11, 2012 at 9:31 PM, Erick Erickson erickerick...@gmail.comwrote: Alexandre: Have you thought about grouping? If you can analyze the incoming documents and include a field such that similar documents map to the

Re: Solr core as a dispatcher

2012-01-11 Thread Hector Castro
Thanks for the reply, Ken – it was your training session that brought the dispatcher core approach to my attention in the first place. Regarding your deep query point, if you're in a situation where numFound=5000 and you're trying to output all 5000 records at once – your point seems to

Acceptable Response Time

2012-01-11 Thread Sujatha Arun
Hello, I am Looking into trigger point for sharding Indexes based on response time ,and would like to define an acceptable response time. Given a 3GB index ,when Can i think of sharding .The response times being variable based on the query and varies from 100ms to 600ms .We are running solr

Re: query parser attaches text: prefix to query

2012-01-11 Thread Matthias Müller
that's it. thanks!

Determining which shard is failing using partialResults / some other technique?

2012-01-11 Thread Gilles Comeau
Hi Solr Users, Does anyone happen to know if the keyword partialResults be used in a solr http request? (partialResults is turned off at the .xml level) Something like: http://server:8080/solr/master/select?distrib=true

Mohanty:RE: Not able to see the output when search is clicked

2012-01-11 Thread Rajdeep Alapati
Hi Mohanty, Can you please guess what is happening on my side. Any help would be great Thanks Raj Deep Alapati From: Gora Mohanty [g...@mimirtech.com] Sent: 11 January 2012 17:15 To: solr-user@lucene.apache.org Subject: Re: Not able to see the output when

Re: exception while loading with DIH multi-threaded

2012-01-11 Thread Bernd Fehling
Hi Mikhail, thanks for pointing me to the issue. Regards, Bernd Am 11.01.2012 21:47, schrieb Mikhail Khludnev: FYI, it's https://issues.apache.org/jira/browse/SOLR-2804 I'm trying to address it. On Wed, Jan 11, 2012 at 5:49 PM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: After

Re: Mohanty:RE: Not able to see the output when search is clicked

2012-01-11 Thread Gora Mohanty
On Thu, Jan 12, 2012 at 12:49 PM, Rajdeep Alapati rajdeep.alap...@benefitfocus.com wrote: Hi Mohanty, Can you please guess what is happening on my side. Any help would be great [...] People have followed up to you in your original thread. Could you please respond there, and provide the