Phrase matching on a text field

2009-05-07 Thread Phil Chadwick
Hi, I'm trying to figure out why phrase matching on a text field only works some of the time. I have a SOLR index containing a document titled FUTURE DIRECTIONS FOR INTEGRATED CATCHMENT. The FOR seems to be causing a problem... The title field is indexed as both s_title and t_title (string and

What are the Unicode encodings supported by Solr?

2009-05-07 Thread KK
Hi, I'd like to know about the different Unicode[/any other?] encodings supported by Solr for posting docs [thru Solrj in my case]. Is it that just UTF-8, UCN supported or other character encodings like NCR(decimal), NCR(hex) etc are supported as well? Now the problem is that during automating

Dataimporthandler Timestamp Error ?

2009-05-07 Thread gateway0
Hi, when I do a full import I get the following error : Caused by: java.sql.SQLException: Cannot convert value '-00-00 00:00:00' from column 10 to TIMESTAMP. at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at

Re: Dataimporthandler Timestamp Error ?

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
you may need to change the mysql connection parameters so that it does not throw error for null date jdbc:mysql://localhost/test?zeroDateTimeBehavior=convertToNull On Thu, May 7, 2009 at 1:39 PM, gateway0 reiterwo...@yahoo.de wrote: Hi, when I do a full import I get the following error :

Re: Creating new QParserPlugin

2009-05-07 Thread Andrey Klochkov
Hi! I agree that Solr is difficult to extend in many cases. We just patch Solr, and I guess many other users patch it too. What I propose is to create some Solr-community site (Solr incubator?) to public patches there, and Solr core team could then look there and choose patches to apply to the

Re: Dataimporthandler Timestamp Error ?

2009-05-07 Thread gateway0
Awesome, thanks!!! I first thought it could be blob-field related. Have a nice day Sebastian Noble Paul നോബിള്‍ नोब्ळ्-2 wrote: you may need to change the mysql connection parameters so that it does not throw error for null date

French and SpellingQueryConverter

2009-05-07 Thread Jonathan Mamou
Hi I have tried to run the following code package org.apache.solr.spelling; import org.apache.lucene.analysis.fr.FrenchAnalyzer; public class Test { public static void main (String args[]) { SpellingQueryConverter sqc = new SpellingQueryConverter(); sqc.analyzer

Re: no subject aka Replication Stall

2009-05-07 Thread Jeff Newburn
We have not pushed the fix into production yet. However, I am wondering two things. 1. If the download takes more than 10 seconds (our replication can take up to 90 seconds) will that be an issue 2. There are 3 patches, 2 have 2 line changes 1 has a large amount. Do we need the latest 2 or just

Re: Solr Plugins Simple Questions for the Simpleton

2009-05-07 Thread Jeff Newburn
On May 6, 2009, at 3:25 PM, Jeff Newburn wrote: We are trying to implement a SearchCompnent plugin. I have been looking at QueryElevateComponent trying to weed through what needs to be done. My basic desire is to get the results back and manipulate them either by altering the actual

Re: Upgrading from 1.2.0 to 1.3.0

2009-05-07 Thread Rob Casson
this isn't advice on how to upgrade, but if you/your-project have a bit of time to wait, 1.4 sounds like it's getting close to an official releasefyi. cheers, rob On Tue, May 5, 2009 at 1:05 PM, Francis Yakin fya...@liquid.com wrote: What's the best way to upgrade solr from 1.2.0 to 1.3.0

Re: Solr autocompletion in rails

2009-05-07 Thread manisha_5
Thanks a lot for the information. But I am still a bit confused about the use of TermsComponents. Like where are we exactly going to put these codes in Solr.For example I changed schema.xml to add autocomplete feauture.I read your blog too, its very helpful.But still a little confused. :-(( Can

Re: When should I optimize/

2009-05-07 Thread Eric Sabourin
Great... thanks for the response! 2009/5/7 Noble Paul നോബിള്‍ नोब्ळ् noble.p...@corp.aol.com it is wise to optimize the index once in a while (daily may be). But it depends on how many commits you do in a day. Every commit causes fragmentation of index files and your search can become slow if

Is it possible to writing solr result on disk from the server side?

2009-05-07 Thread arno13
Do you know if it's possible to writing solr results directly on a hard disk from server side and not to use an HTTP connection to transfer the results? While the query time is very fast for solr, I want to do that, cause of the time taken during the transfer of the results between the client

Re: no subject aka Replication Stall

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
the patches have gone into the trunk. The latest patch should be the one if you wish to run a patched Solr. 10 secs readTimeout means that if there is no data coming from the other end for 10 secs, then the waiting thread returns throwing an exception. It is not the total time taken to read the

Re: Is it possible to writing solr result on disk from the server side?

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
did you consider using an EmbeddedSolrServer? On Thu, May 7, 2009 at 8:25 PM, arno13 arnaud.gaudi...@healthonnet.org wrote: Do you know if it's possible to writing solr results directly on a hard disk from server side and not to use an HTTP connection to transfer the results? While the query

Re: no subject aka Replication Stall

2009-05-07 Thread Jeff Newburn
Excellent! Thank you I am going to start testing that. -- Jeff Newburn Software Engineer, Zappos.com jnewb...@zappos.com - 702-943-7562 From: Noble Paul നോബിള്‍ नोब्ळ् noble.p...@corp.aol.com Reply-To: solr-user@lucene.apache.org Date: Thu, 7 May 2009 20:26:02 +0530 To:

Re: large index vs multicore

2009-05-07 Thread Nicolas Pastorino
Hi, and sorry for slightly hijacking the thread, On Mar 26, 2009, at 2:54 , Otis Gospodnetic wrote: Hi, Without knowing the details, I'd say keep it in the same index if the additional information shares some/enough fields with the main product data and separately if it's sufficiently

RE: When should I optimize/

2009-05-07 Thread Wang, Ching-Hsien
We do optimize once a day at 1am. Ching-hsien Wang,  Manager Library and Archives System Support Branch Office of Chief Information Officer Smithsonian Institution 202-633-5581(office)  202-312-2874(fax) wan...@si.edu Visit us online: www.siris.si.edu -Original Message- From: Eric

Re: Phrase matching on a text field

2009-05-07 Thread Jay Hill
The string fieldtype is not being tokenized, while the text fieldtype is tokenized. So the stop word for is being removed by a stop word filter, which doesn't happen with the text field type (no tokenizing). Have a look at the schema.xml in the example dir and look at the default configuration

Re: French and SpellingQueryConverter

2009-05-07 Thread Jay Hill
It seems to me that this is just the expected behavior of the FrenchAnalyzer using the FrenchStemmer. I'm not familiar with the French language, but in English words like running, runner, and runs are all stemmed down to run as intended. I don't know what other words in French would stem down to

RE: What are the Unicode encodings supported by Solr?

2009-05-07 Thread Steven A Rowe
Hi KK, On 5/7/2009 at 2:55 AM, KK wrote: In some of the pages I'm getting some \ufffd chars which I think is some sort of unmappable[by Java?] character, right?. Any idea on how to handle this? Just replacing with blank char will not do [this depends on the requirement, though]. From

Sorting by 'starts with'

2009-05-07 Thread wojtekpia
I have an index of product names. I'd like to sort results so that entries starting with the user query come first. E.g. q=kitchen Results would sort something like: 1. kitchen appliance 2. kitchenaid dishwasher 3. fridge for kitchen It looks like using a query Function Query comes close,

Solr spring application context error

2009-05-07 Thread Raju444us
I have configured solr using tomcat.Everything works fine.I overrode QParserPlugin and configured it.The overriden QParserPlugin has a dependency on another project say project1.So I made a jar of the project and copied the jar to the solr/home lib dir. the project1 project is using spring.It

Re: solrcofig.xml - need some info

2009-05-07 Thread Raju444us
This is resolved.I solved this by reading solrPlugins on the solr wiki. Thanks, Raju Raju444us wrote: Hi Hoss, If i extend SolrQueryParser and override method getFieldQuery for some customization.Can I configure my new queryParser somthing like below requestHandler name=standard

Re: Control segment size

2009-05-07 Thread vivek sar
Thanks Otis. I did set the maxMergeDocs to 10M, but I still see couple of index files over 30G which do not match with max number of documents. Here are some numbers, 1) My total index size = 66GB 2) Number of total documents = 200M 3) 1M doc = 300MB 4) 10M doc should be roughly around 3-4GB.

Backups using Java-based Replication (forced snapshot)

2009-05-07 Thread Grant Ingersoll
On the page http://wiki.apache.org/solr/SolrReplication, it says the following: Force a snapshot on master.This is useful to take periodic backups .command : http://master_host:port/solr/replication? command=snapshoot This then puts the snapshot under the data directory. Perfectly

Re: French and SpellingQueryConverter

2009-05-07 Thread Jonathan Mamou
Hi It does not seem to be related to FrenchStemmer, the stemmer does not split a word into 2 words. I have checked with other words and SpellingQueryConverter always splits words with special character. I think that the issue is in SpellingQueryConverter class

Autocommit blocking adds? AutoCommit Speedup?

2009-05-07 Thread Jim Murphy
Question 1: I see in DirectUpdateHandler2 that there is a read/Write lock used between addDoc and commit. My mental model of the process was this: clients can add/update documents until the auto commit threshold was hit. At that point the commit tracker would schedule a background commit.

Re: Autocommit blocking adds? AutoCommit Speedup?

2009-05-07 Thread Yonik Seeley
On Thu, May 7, 2009 at 5:03 PM, Jim Murphy jim.mur...@pobox.com wrote: Question 1: I see in DirectUpdateHandler2 that there is a read/Write lock used between addDoc and commit. My mental model of the process was this: clients can add/update documents until the auto commit threshold was hit.  

preImportDeleteQuery

2009-05-07 Thread wojtekpia
Hi, I'm importing data using the DIH. I manage all my data updates outside of Solr, so I use the full-import command to update my index (with clean=false). Everything works fine, except that I can't delete documents easily using the DIH. I noticed the preImportDeleteQuery attribute, but doesn't

Re: Solr autocompletion in rails

2009-05-07 Thread Matt Weber
First, your solrconfig.xml should have the something similar to the following: searchComponent name=termsComp class=org.apache.solr.handler.component.TermsComponent/ requestHandler name=/autoSuggest class=org.apache.solr.handler.component.SearchHandler arr name=components

Re: preImportDeleteQuery

2009-05-07 Thread Martin Davidsson
On May 7, 2009, at 4:52 PM, wojtekpia wrote: Hi, I'm importing data using the DIH. I manage all my data updates outside of Solr, so I use the full-import command to update my index (with clean=false). Everything works fine, except that I can't delete documents easily using the DIH. I

bug? No highlighting results with dismax and q.alt=*:*

2009-05-07 Thread Peter Wolanin
For the Drupal Apache Solr Integration module, we are exploring the possibility of doing facet browsing - since we are using dismax as the default handler, this would mean issuing a query with an empty q and falling back to to q.alt='*:*' or some other q.alt that matches all docs. However, I

Re: Autocommit blocking adds? AutoCommit Speedup?

2009-05-07 Thread Jim Murphy
Interesting. So is there a JIRA ticket open for this already? Any chance of getting it into 1.4? Its seriously kicking out butts right now. We write into our masters with ~50ms response times till we hit the autocommit then add/update response time is 10-30 seconds. Ouch. I'd be willing to

Re: Phrase matching on a text field

2009-05-07 Thread Phil Chadwick
Hi Jay Thank you for your response. The data relating to the string (s_title) defines *exactly* what was fed into the SOLR indexing. The string is not otherwise relevant to the question. The essence of my question is why can the indexed text (t_title) not be phrase matched by the query on the

StatsComponent and 1.3

2009-05-07 Thread David Shettler
Foreword: I'm not a java developer :) OSVDB.org and datalossdb.org make use of solr pretty extensively via acts_as_solr. I found myself with a real need for some of the StatsComponent stuff (mainly the sum feature), so I pulled down a nightly build and played with it. StatsComponent proved

Re: Autocommit blocking adds? AutoCommit Speedup?

2009-05-07 Thread Yonik Seeley
On Thu, May 7, 2009 at 8:37 PM, Jim Murphy jim.mur...@pobox.com wrote: Interesting.  So is there a JIRA ticket open for this already? Any chance of getting it into 1.4? No ticket currently open, but IMO it could make it for 1.4.  Its seriously kicking out butts right now.  We write into our

Re: Backups using Java-based Replication (forced snapshot)

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
makes sense. I'll open an issue On Fri, May 8, 2009 at 1:53 AM, Grant Ingersoll gsing...@apache.org wrote: On the page http://wiki.apache.org/solr/SolrReplication, it says the following: Force a snapshot on master.This is useful to take periodic backups .command :  

Re: preImportDeleteQuery

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
are you doing a full-import or a delta-import? for delta-import there is an option of deletedPkQuery which should meet your needs On Fri, May 8, 2009 at 5:22 AM, wojtekpia wojte...@hotmail.com wrote: Hi, I'm importing data using the DIH. I manage all my data updates outside of Solr, so I use

Re: Solr spring application context error

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
a point to keep in mind is that all the plugin code and everything else must be put into the solrhome/lib directory. where have you placed the file com/mypackage/applicationContext.xml ? On Fri, May 8, 2009 at 12:19 AM, Raju444us gudipal...@gmail.com wrote: I have configured solr using

Fwd: Solr MultiCore dataDir bug - a fix

2009-05-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
I didn't notice that the mail was not sent to the list. Plz send all your communication to the mailing list -- Forwarded message -- From: Noble Paul നോബിള്‍ नोब्ळ् noble.p...@corp.aol.com Date: 2009/5/8 Subject: Re: Solr MultiCore dataDir bug - a fix To:

Re: Phrase matching on a text field

2009-05-07 Thread Phil Chadwick
Hi, I have tracked this problem to: https://issues.apache.org/jira/browse/SOLR-879 Executive summary is that there are errors that relate to text fields in both: - src/java/org/apache/solr/search/SolrQueryParser.java - example/solr/conf/schema.xml It is fixed in 1.4. Thank you Yonik

RE: Core Reload issue

2009-05-07 Thread Sagar Khetkade
From my understanding re-indexing the documents is a different thing. If you have the stop word filter for field type say text then after reloading the core if i type in a query which is stop word only it would get parsed from stop word filter which would eventually will not serach against