parsing many documents takes too long

2011-08-12 Thread Tri Nguyen
Hi,   My results from solr returns about 982 documents and I use jaxb to parse them into java objects, which takes about 469 ms, which is over my 150-200ms threshold.   Is there a solution around this?  Can I store the java objects in the index and return them in the solr response and then

Re: Strip special chars like -

2011-08-12 Thread roySolr
Erick, you're right. It's working, my schema looks like this: fieldType name=name_type class=solr.TextField positionIncrementGap=100 analyzer type=index charFilter class=solr.HTMLStripCharFilterFactory/ tokenizer class=solr.WhitespaceTokenizerFactory/ filter

Clustering not working when using 'text' field as snippet.

2011-08-12 Thread Pablo Queixalos
Hi, I am using solr-3.3.0 and carrot² clustering which works fine out of the box with the examples doc and default solr configuration (the 'features' Field is used as snippet). I indexed my own documents using the embed ExtractingRequestHandler wich by default stores contents in the

Last successful build of Solr 4.0 and Near Realtime Search

2011-08-12 Thread Vadim Kisselmann
Hi folks, I'm writing here again (beside Jira: SOLR-2565), eventually any one can help here: I tested the nightly build #1595 with an new patch (2565), but NRT doesn't work in my case. I index 10 docs/sec, it takes 1-30sec. to see the results. same behavior when i update an existing document.

Re: Clustering not working when using 'text' field as snippet.

2011-08-12 Thread Stanislaw Osinski
Hi Pablo, The reason clustering doesn't work with the text field is that the field is not stored: field name=text type=text_general indexed=true stored=false multiValued=true/ For clustering to work, you'll need to keep your documents' titles and content in stored fields. Staszek On Fri,

Re: SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.ICUTokenizerFactory'

2011-08-12 Thread Péter Király
Hi Satish, : I also added the following files to my apache-solr-3.3.0\example\lib : folder: I use ICU, and I copied the jar files not into example/lib as you did, but example/solr/lib. First I had to create that directory. It works for me both under 3.1, 3.2 and 3.3. In multicore setup I

Re: Timeout trying to index from nutch

2011-08-12 Thread Markus Jelsma
Firewall? Proxy? I am new user and I have SOLR installed. I can use the admin page and query the example data. However, I was using nutch to load index with intranet web pages and I got this message. SolrIndexer: starting at 2011-08-12 16:52:44

SOLR 3.3.0 multivalued field sort problem

2011-08-12 Thread johnnyisrael
Hi, I am currently using SOLR 1.4.1, With this version sorting working fine even in multivalued field. Now I am planning to upgrade my SOLR version from 1.4.1 -- 3.3.0, In this latest version sorting is not working on multivauled field. So I am in unable to upgrade my SOLR due to this drawback.

Fuzzy search with sort combination - drawback

2011-08-12 Thread johnnyisrael
Hi, I am having one problem while using the fuzzy search from query. I have two fields in my SOLR output, one field is endNgramed and other one is a normal Integer field which will have my customized score for that document. I have a handler[myhandler] which by default will sort the documents

Re: SOLR 3.3.0 multivalued field sort problem

2011-08-12 Thread Péter Király
Hi, There is no direct solution, you have to create single value field(s) to create search. I am aware of two workarounds: - you can use a random or a given (e.g. the first) instance of the multiple values of the field, and that would be your sortable field. - you can create two sortable fields:

RE: Clustering not working when using 'text' field as snippet.

2011-08-12 Thread Pablo Queixalos
Thanks for your reply Staszek, Of course, the field has to be stored. I forgot to mention that I already updated the schema for that. I also checked that data was effectiveley stored in that field. Anyway, I tried to reproduce it on a fresh Solr install and clustering works well. ;-)

Not update on duplicate key

2011-08-12 Thread Rohit
Hi All, Please correct me if I am wrong, but when I am trying to insert a document into Solr which was previously index, it overwrites the current key. Is there a way to change the behaviour, 1. I don't want Solr to override but on the other hand it should ignore the entry 2. Also,

Post content to be indexed to Solr

2011-08-12 Thread rahul
Hi, Currently I am indexing documents by directly adding files as 'req.addFile(fi);' or by sending the content of the file like 'req.addContentStream(stream);' using solrj. Assume, if the solrj client Solr server are in different network (ie, Solr server is in remote location) I need to

Nutch related issue: URL Ignore

2011-08-12 Thread Pawan Darira
hi i am using nutch 1.2. in my crawl-urlfilter.txt, i am specifying URLs to be skipped. i am giving some patterns that need to be skipped but it is not working e.g. -^http://([a-z0-9]*\.)*domain.com +^http://([a-z0-9]*\.)*domain.com/([0-9-a-z])*.html -^http://([a-z0-9]*\.)*domain.com/([a-z/])*

Re: Unbuffered entity enclosing request can not be repeated Invalid chunk header

2011-08-12 Thread Vadim Kisselmann
Hi Markus, thanks for your answer. I'm using Solr. 4.0 and jetty now and observe the behavior and my error logs next week. tomcat can be a reason, we will see, i'll report. I'm indexing WITHOUT batches, one doc after another. But i would try out the batch indexing as well as retry indexing

sorting issue with solr 3.3

2011-08-12 Thread Bernd Fehling
It turned out that there is a sorting issue with solr 3.3. As fas as I could trace it down currently: 4 docs in the index and a search for *:* sorting on field dccreator_sort in descending order

Re: how to integrate solr with web page?

2011-08-12 Thread Ahmet Arslan
Hi i have queried solr to retrieve information from database now i have to integrate with web page...i dont know how to implement this please help me...     actually i have one jsp page which is having search field and search button now i need to get the results from solr in the jsp

JEE servlet mapping, security and multiple Solr cores

2011-08-12 Thread Jaeger, Jay - DOT
This is both an FYI for the list so the issue gets documented and a suggestion for the developers. I thought about a JIRA, and would be happy to submit one, but the issue is pretty environment-specific, so I have not done so at this point. In testing Solr 3.3 under WebSphere Application

Re: how to integrate solr with web page?

2011-08-12 Thread Nicholas Chase
On 8/12/2011 12:52 AM, nagarjuna wrote: Hi i have queried solr to retrieve information from database now i have to integrate with web page...i dont know how to implement this please help me... actually i have one jsp page which is having search field and search button now i need to get the

Re: SOLR 3.3.0 multivalued field sort problem

2011-08-12 Thread Martijn v Groningen
Hi Johnny, Sorting on a multivalued field has never really worked in Solr. Solr versions = 1.4.1 allowed it, but there was a change that an error occurred and that the sorting might not be what you expect. From Solr 3.1 and up sorting on a multivalued isn't allowed and a http 400 is returned.

Exception DirectSolrSpellChecker when using spellcheck.q

2011-08-12 Thread O. Klein
Spellchecker works fine, but when using spellcheck.q it gives following exception (queryAnalyzerFieldType is defined if that would matter). Is it bug or am I doing something wrong? 2011-08-12 17:30:54,368 java.lang.NullPointerException at

Tomcat7 with Solr closes at fixed hours, every time another hour

2011-08-12 Thread Adrian Fita
Hello. I'm having a Solr running within Tomcat7 and Tomcat is closing at fixed hours, everytime is a different hour. catalina.log doesn't show anything other than a clean tomcat shutdown (no exception or anything). I would really apreciate some advice on how to debug this. Tomcat doesn't run

Re: sorting issue with solr 3.3

2011-08-12 Thread Yonik Seeley
On Fri, Aug 12, 2011 at 9:53 AM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: It turned out that there is a sorting issue with solr 3.3. As fas as I could trace it down currently: 4 docs in the index and a search for *:* sorting on field dccreator_sort in descending order

Re: LockObtainFailedException

2011-08-12 Thread Naveen Gupta
HI Peter I found the issue, Actually we were getting this exception because of JVM space. I allocated 512 xms and 1024 xmx .. finally increased the time limit for write lock to 20 secs .. things are working fine ... but still it did not help ... On closely analysis of doc which we were

Re: Need help indexing/querying a particular type of hierarchy

2011-08-12 Thread Michael B. Klein
After a whole lot of facet-wrangling, I've come up with a practical solution that suits my situation, which is to index each triple as a series of paths. For example, if the shelve process of the accessionWF workflow is completed, it gets indexed as: field name=wf_wpsaccessionWF/field field

Re: sorting issue with solr 3.3

2011-08-12 Thread Yonik Seeley
On Fri, Aug 12, 2011 at 1:04 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Fri, Aug 12, 2011 at 9:53 AM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: It turned out that there is a sorting issue with solr 3.3. As fas as I could trace it down currently: 4 docs in the index and a

custom velocity tool

2011-08-12 Thread Stéphane Campinas
Hi, I am working with the velocity response writer, and I want to develop a custom velocity tool. To do so, I have written a JAVA class that looks like that: @DefaultKey(mytool) public class MyCustomTool { public MyCustomTool() { } public String doit(Object arg) {

Re: sorting issue with solr 3.3

2011-08-12 Thread Yonik Seeley
On Fri, Aug 12, 2011 at 2:08 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Fri, Aug 12, 2011 at 1:04 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Fri, Aug 12, 2011 at 9:53 AM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: It turned out that there is a sorting issue with

Some questions about SolrJ

2011-08-12 Thread Shawn Heisey
I currently have a build system for my Solr index written in Perl. I am in the process of rewriting it in Java. I've reached the part of the project where I'm using SolrJ, and I have a bunch of questions. All of the SolrJ examples I can find are too simple to answer them. A note before I

Re: Some questions about SolrJ

2011-08-12 Thread Shawn Heisey
On 8/12/2011 1:49 PM, Shawn Heisey wrote: I am sure that I have more questions, but I may be able to answer a lot of them myself if I can see better examples. Thought of another question. My Perl build system uses DIH for all indexing, but with the Java rewrite I am planning to do all

Re: sorting issue with solr 3.3

2011-08-12 Thread Yonik Seeley
I've checked in an improved TestSort that adds deleted docs and randomizes things a lot more (and fixes the previous reliance on doc ids not being reordered). I still can't reproduce this error though. Is this stock solr? Can you verify that the documents are in the wrong order also (and not just

dataimporthandler large dataset

2011-08-12 Thread Eric Myers
Recently started looking into solr to solve a problem created before my time. We have a dataset consisting of 390,000,000+ records that had a search written for it using a simple query. The problem is that the dataset needs additional indices to keep operating. The DBA says no go, too large a

Re: dataimporthandler large dataset

2011-08-12 Thread Kyle Lee
We have a 200,000,000 record index with 14 fields, and we can re-index the entire data set in about five hours. One thing to note is that the DataImportHandler uses one thread per entity by default. If you have a multcore box, you can drastically speed indexing by specifying a threadcount of n+1,

Re: dataimporthandler large dataset

2011-08-12 Thread Shawn Heisey
On 8/12/2011 3:32 PM, Eric Myers wrote: Recently started looking into solr to solve a problem created before my time. We have a dataset consisting of 390,000,000+ records that had a search written for it using a simple query. The problem is that the dataset needs additional indices to keep

sorting distance in solr 1.4.1

2011-08-12 Thread Tri Nguyen
Hi,   We are using solr 1.4.1 and we need to sort our results by distance. We have lat lons for each document in the response and our reference point.   Is it possible?  I read about the spatial plugin but the does range searching:  

Re: custom velocity tool

2011-08-12 Thread Erik Hatcher
Stephane - Also - I don't think even with v.properties=velocity.properties that it'd be picked up from the solr-home/conf directory the way the code is loading it using SolrResourceLoader. The .properties file would need to be in your JAR file for your custom tool (or in the classpath somehow

RE: need some guidance about how to configure a specific solr solution.

2011-08-12 Thread Jonathan Rochkind
I don't know anything about LifeRay (never heard of it), but it sounds like you've actually figured out what you need to know about LifeRay, all you've got left is: how to replicate the writer solr server content into the readers. This should tell you how: