Re: Query/Delete performance difference between straight HTTP and SolrJ

2011-10-26 Thread Michael Kuhlmann
Hi, Am 25.10.2011 23:53, schrieb Shawn Heisey: On 10/20/2011 11:00 AM, Shawn Heisey wrote: [...] I've noticed a performance discrepancy when processing every one of my delete records, currently about 25000 of them. I din't understand what a delete record is. Do you delete records in Solr?

Re: some basic information on Solr

2011-10-26 Thread stockii
i think with incident he mean, failures / downtimes / problems with solr !? - --- System One Server, 12 GB RAM, 2 Solr Instances, 8 Cores, 1 Core with 45 Million Documents other Cores 200.000 - Solr1 for Search-Requests

Re: java.net.SocketException: Too many open files

2011-10-26 Thread Michael Kuhlmann
Hi; we have a similar problem here. We already raised the file ulimit on the server to 4096, but this only defered the problem. We get a TooManyOpenFilesException every few months. The problem has nothing to do with real files. When we had the last TooManyOpenFilesException, we investigated with

Re: RE: Dismax handler - whitespace and special character behaviour

2011-10-26 Thread Khorail
In fact I tried without WordDelimiterFilterFactory (using a PatternTokenizerFactory to tokenize on special chars) and I still have the same problem. Apparently dismax handler thinks that 'france-histoire' is a single word even if I tokenize on '-' Le , Demian Katz demian.k...@villanova.edu

Saravanan Chinnadurai/Actionimages is out of the office.

2011-10-26 Thread Saravanan . Chinnadurai
I will be out of the office starting 26/10/2011 and will not return until 28/10/2011. Please email to itsta...@actionimages.com for any urgent issues. Action Images is a division of Reuters Limited and your data will therefore be protected in accordance with the Reuters Group Privacy / Data

solr.PatternReplaceFilterFactory AND endoffset

2011-10-26 Thread roySolr
Hi, I have some problems with the patternreplaceFilter. I can't use the worddelimiter because i only want to replace some special chars given by myself. Some example: Tottemham-hotspur (london) Arsenal (london) I want this: replace - with ( OR ) with . In the analytics i see this:

Re: [ANNOUNCEMENT] PHP Solr Extension 1.0.1 Stable Has Been Released

2011-10-26 Thread alex
Hello roySolr, roySolr wrote: Are you working on some changes to support earlier versions of PHP? What is the status? I have supplied a patch, so that it can be compiled with PHP 5.2: https://bugs.php.net/bug.php?id=59808 https://bugs.php.net/bug.php?id=59808 I contacted Israel a while

Re: Incorrect Search Results showing up

2011-10-26 Thread Grant Ingersoll
If you add debugQuery=true to your request, what does it show for that last result? On Oct 25, 2011, at 5:31 PM, aronitin wrote: Hi Group, I've the defined a type text in the SOLR schema as shown below. fieldType name=text class=solr.TextField positionIncrementGap=100

Re: questions about autocommit committing documents

2011-10-26 Thread Erick Erickson
Not sure what you mean by a callback, can you clarify? You don't get anything except the return from the add call as far as I know... Best Erick On Tue, Oct 25, 2011 at 4:15 AM, darul daru...@gmail.com wrote: I was not sure thank you. -- View this message in context:

MultiValued fields and Facets...

2011-10-26 Thread Tiernan OToole
Good morning all. I am currently indexing about 11 million records, and would like to add facating to the results page. I have tweaked the query string to include facating, but i am not getting anything back. an Example Query string (slightly modified) is as follows:

Re: A sort-by-geodist question

2011-10-26 Thread Erick Erickson
Hmmm, I'm not sure this is supported. Why can't you just use the location type provided in the example schema? Best Erick On Mon, Oct 24, 2011 at 9:39 PM, ☼ 林永忠 ☼ (Yung-chung Lin) henearkrx...@gmail.com wrote: Hi, I've started to use Solr to build up a search service, but I have encountered

Too many values for UnInvertedField faceting on field autocompleteField

2011-10-26 Thread Torsten Krah
I am getting this SolrException Too many values for UnInvertedField faceting on field autocompleteField. Already added facet.method=enum to my search handler definition but still this exception does happen. Any known fix or workaround whan i can do to get a result? regards Torsten smime.p7s

Re: A sort-by-geodist question

2011-10-26 Thread Yung-chung Lin
Hi, Thanks for the reply. I switched to the location type. And it's working now. Am not sure if it's a problem with geohash or it's because I don't know well about the configurations, but it works now. Thanks for the reply. Yung-chung Lin 2011/10/26 Erick Erickson erickerick...@gmail.com

Re: MultiValued fields and Facets...

2011-10-26 Thread Erik Hatcher
That URL has several oddities to it... empty fq and qt parameters. Try simply ?q=*:*facet=onfacet.field=Categoryfacet.field=Warehouse and see if that helps. Erik On Oct 26, 2011, at 07:08 , Tiernan OToole wrote: Good morning all. I am currently indexing about 11 million records,

Re: MultiValued fields and Facets...

2011-10-26 Thread Tiernan OToole
Ok, so now i am getting something back, but still getting odd results... I actually made a mistake in the first question... Category is MultiValued, but Warehouse is not... So, when i run the query, you sugested, Category comes back with facets and counts, which is one step closer to where i want

Re: Too many values for UnInvertedField faceting on field autocompleteField

2011-10-26 Thread Yonik Seeley
On Wed, Oct 26, 2011 at 7:39 AM, Torsten Krah tk...@fachschaft.imn.htwk-leipzig.de wrote: I am getting this SolrException Too many values for UnInvertedField faceting on field autocompleteField. Already added facet.method=enum to my search handler definition but still this exception does

Re: NRT and replication

2011-10-26 Thread Esteban Donato
thanks Mark and Tomas. Tomas, you mean doing soft commits to all the slave nodes right? If so, that is what I'm planning to do with the update processor commented above. 2011/10/21 Tomás Fernández Löbbe tomasflo...@gmail.com: I was thinking in this, would it make sense to keep the master /

RE: Replication issues with multiple Slaves

2011-10-26 Thread Jaeger, Jay - DOT
Thanks for that information. It was most useful. Does anyone know: when this happens does the slave continue using its old index, and then try again at the next time interval? (I sure hope so). JRJ -Original Message- From: Markus Jelsma [mailto:markus.jel...@openindex.io] Sent:

RE: Loading data to SOLR first time ( taking too long)

2011-10-26 Thread Jaeger, Jay - DOT
No, we do not use DIH. Based on other responses I saw, its seems likely that the issue is in the DIH component somehow. JRJ -Original Message- From: Awasthi, Shishir [mailto:shishir.awas...@baml.com] Sent: Tuesday, October 25, 2011 3:24 PM To: solr-user@lucene.apache.org; Jaeger, Jay

missing core name in path

2011-10-26 Thread Fred Zimmerman
It is not a multi-core setup. The solr.xml has null value for cores. ? HTTP ERROR 404 Problem accessing /solr/admin/index.jsp. Reason: missing core name in path 2011-10-26 13:40:21.182:WARN::/solr/admin/ java.lang.IllegalStateException: STREAM at

Re: data import in 4.0

2011-10-26 Thread Adeel Qureshi
Any comments .. please I am able to do the bulkimport without nested query but with nested query it just keeps working on it and never seems to end .. I would appreciate any help Thanks Adeel On Sat, Oct 22, 2011 at 11:12 AM, Adeel Qureshi adeelmahm...@gmail.comwrote: yup that was it .. my

fixed schema problems, now running out of memory?

2011-10-26 Thread Fred Zimmerman
It's a small indexing job coming from nutch. 2011-10-26 15:07:29,039 WARN mapred.LocalJobRunner - job_local_0011 java.io.IOException: org.apache.solr.client.solrj.SolrServerException: Error executi$ at org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat.getRec$ at

Re: Query/Delete performance difference between straight HTTP and SolrJ

2011-10-26 Thread Shawn Heisey
On 10/26/2011 1:30 AM, Michael Kuhlmann wrote: Hi, Am 25.10.2011 23:53, schrieb Shawn Heisey: On 10/20/2011 11:00 AM, Shawn Heisey wrote: [...] I've noticed a performance discrepancy when processing every one of my delete records, currently about 25000 of them. I din't understand what a

Re: Query/Delete performance difference between straight HTTP and SolrJ

2011-10-26 Thread Shawn Heisey
On 10/26/2011 10:29 AM, Shawn Heisey wrote: One possible thing I can do to make the Java code even faster is to set rows to zero before doing the query, since I only need numFound, not the actual results. The Perl code does NOT do this, and yet it's super fast. It turns out I already

Java 7u1 fixes index corruption and crash bugs in Apache Lucene Core and Apache Solr

2011-10-26 Thread Uwe Schindler
Hi users of Apache Lucene Core and Apache Solr, Oracle released Java 7u1 [1] on October 19. According to the release notes and tests done by the Lucene committers, all bugs reported on July 28 are fixed in this release, so code using Porter stemmer no longer crashes with SIGSEGV. We were not able

Re: fixed schema problems, now running out of memory?

2011-10-26 Thread Fred Zimmerman
More on what's happening. It seems to be timing out during the commit. The new documents are small, but the existing index is large (11 million records). INFO: Closing Searcher@4a7df6 main

Re: fixed schema problems, now running out of memory?

2011-10-26 Thread Fred Zimmerman
http://wiki.apache.org/solr/SolrPerformanceFactors#Schema_Design_Considerations The number of indexed fields greatly increases the following: - Memory usage during indexing - Segment merge time - Optimization times - Index size These impacts can be reduced by the use of

Upgratding the Index from 1.4.1 to 3.4 using replication

2011-10-26 Thread Nemani, Raj
All, We are planning to upgrade our Solr instance from 1.4.1 to 3.4. We understand that we need to re-index all the documents given the changes to the index structure. If we setup a replication pipe with 1.4.1 as the Master and 3.4 as the salve (with an empty index) is there would the

Can dynamic fields defined by a prefix be used with LatLonType?

2011-10-26 Thread Tom Cooke
Hi, I'm adding support for lat/lon data into an existing schema which uses prefix-based dynamic fields e.g. OBJECT_I_*. I would like to add OBJECT_LL_* as a dynamic field for LatLonType data but it seems that the LatLonType always needs to add suffixes for the dynamically created subfields

solr break up word

2011-10-26 Thread Boris Quiroz
Hi, I've solr running on a CentOS server working OK, but sometimes my application needs to index some parts of a word. For example, if I search 'dislike' word fine but if I search 'disl' it returns zero. Also, if I search 'disl*' returns some values (the same if I search for 'dislike') but if

Get results ordered by field content starting with specific word

2011-10-26 Thread darul
I have seen many threads talking about it but not found any way on how to resolve it. In my schema 2 fields : Results are sorted by field2 desc like in the following listing when looking for word1 as query pattern: I would like to get Doc3 at the end because word1 is not at the beginning of

RE: some basic information on Solr

2011-10-26 Thread Jaeger, Jay - DOT
It didn't look like that, but maybe. Our experience has been very very good. I don't think we have seen a crash in our prototype to date (though that prototype is also not very busy). We have had as many a four cores, with as many as 35 million documents. -Original Message- From:

RE: Difficulties Installing Solr with Jetty 7.x

2011-10-26 Thread Jaeger, Jay - DOT
From your logs, it looks like the Solr library is being found just fine, and that the servlet is initing OK. Does your Jetty configuration specify index.jsp in a welcome list? We had that problem in WebSphere: we got 404's the same way, and the cure was to modify the Jetty web.xml to include:

RE: Difficulties Installing Solr with Jetty 7.x

2011-10-26 Thread Jaeger, Jay - DOT
ERRATA, that should the the *SOLR* web.xml (not the Jetty web.xml) Sorry for the confusion. -Original Message- From: Jaeger, Jay - DOT [mailto:jay.jae...@dot.wi.gov] Sent: Wednesday, October 26, 2011 4:02 PM To: 'solr-user@lucene.apache.org' Subject: RE: Difficulties Installing Solr with

RE: Upgratding the Index from 1.4.1 to 3.4 using replication

2011-10-26 Thread Jaeger, Jay - DOT
I very much doubt that would work: different versions of Lucene involved, and Solr replication does just a streamed file copy, nothing fancy. JRJ -Original Message- From: Nemani, Raj [mailto:raj.nem...@turner.com] Sent: Wednesday, October 26, 2011 12:55 PM To:

Re: DisMax search

2011-10-26 Thread Erik Hatcher
Maybe a case sensitive issue? defType it should be. Erik On Oct 26, 2011, at 16:03, jyn7 jyotsna.namb...@gmail.com wrote: Hi, I am using a dismax search and limiting the query parameters using qf: /solrbgp/select/?facet=trueqf=memnumq=%229065%22deftype=dismaxstart=0rows=10 My

Analyzers from schema.xml with custom parser

2011-10-26 Thread Milan Dobrota
I created a custom plugin parser, and it seems like it is ignoring analyzers from schema.xml. Is there any way to associate the two?

Re: Difficulties Installing Solr with Jetty 7.x

2011-10-26 Thread Scott Vanderbilt
Jay: Thanks for the response. $JETTY_HOME/etc/webdefault.xml is the unmodified file that came with Jetty, and it has a welcome-file-list referencing index.jsp, index.html, and index.htm. Attempting to load /solr/admin.index.jsp generates a 404. All other URLs generate a 404 also, except /,

exact matches are not filtered to the top

2011-10-26 Thread Ji, Jason
Hi guys, We have a case that we need to do wildcard search for either user's realname or username.(note that realname is not mandatory) So we specified the copyField as below: field name=username type=text indexed=true stored=true required=true/ field name=realname type=text indexed=true

Re: help needed on solr-uima integration

2011-10-26 Thread Xue-Feng Yang
Hi, Is there logging for uima? From Logging in Solr Admin page, I couldn't find it. Thanks, Xue-Feng From: Xue-Feng Yang just4l...@yahoo.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org Sent: Tuesday, October 25, 2011 8:50:05 PM Subject: Re:

Re: help needed on solr-uima integration

2011-10-26 Thread Xue-Feng Yang
Hi, From Solr Info page, I can see my solr-uima core is there, but updateRequestProcessorChain is not there. What is the reason? Thanks, Xue-Feng From: Xue-Feng Yang just4l...@yahoo.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org Sent:

Re: Query/Delete performance difference between straight HTTP and SolrJ

2011-10-26 Thread Michael Sokolov
Have you checked to see when you are committing? Is the pattern the same in both instances? If you are committing after each delete request in Java, but not in Perl, that could slow things down. On 10/25/2011 5:53 PM, Shawn Heisey wrote: On 10/20/2011 11:00 AM, Shawn Heisey wrote: I've got

Re: Query/Delete performance difference between straight HTTP and SolrJ

2011-10-26 Thread Shawn Heisey
On 10/26/2011 6:16 PM, Michael Sokolov wrote: Have you checked to see when you are committing? Is the pattern the same in both instances? If you are committing after each delete request in Java, but not in Perl, that could slow things down. The commit happens separately, not during the

Re: DisMax search

2011-10-26 Thread jyn7
I am searching for 9065 , so its not about case sensitivity. My search is searching across all the field names and not limiting it to one field(specified in the qf param and using deftype dismax) -- View this message in context: