Re: mergeFactor / indexing speed

Hi Avlesh, hi Otis, hi Grant, hi all, (enumerating to keep track of all the input) a) mergeFactor 1000 too high I'll change that back to 10. I thought it would make Lucene use more RAM before starting IO. b) ramBufferSize: OK, or maybe more. I'll keep that in mind. c) solrconfig.xml -

Functions in search result

Solr people, Can i retrieve results from a function query? For instance, i have a schema in which all documents have a size in bytes field. For each query, i also need to sum of the bytes field for the returned documents. I know i can use SUM as part of a function query but i cannot figure it

Re: How to configure Solr in Glassfish ?

On 7/20/09 11:08 PM, huenzhao wrote: Yes, I don't know how set solr.home in glassfish with centOS. I tried to configure the solr.home, but the error log is:looking for solr.xml: /var/deploy/solr/solr.xml Is that the appropriate path for your solr.home? What did you intend to set it to?

Re: Rotating the primary shard in /solr/select

On Wed, Jul 29, 2009 at 2:57 AM, Phillip Farber pfar...@umich.edu wrote: Is there any value in a round-robin scheme to cycle through the Solr instances supporting a multi-shard index over several machines when sending queries or is it better to just pick one instance and stick with it. I'm

Re: Rotating the primary shard in /solr/select

On Tue, Aug 4, 2009 at 11:26 AM, Rahul R rahul.s...@gmail.com wrote: Philip, I cannot answer your question, but I do have a question for you. Does aggregation happen at the primary shard ? For eg : if I have three JVMs JVM 1 : My application powered by Solr JVM 2 : Shard 1 JVM 3 : Shard 2

Re: Rotating the primary shard in /solr/select

*The SolrServer is initialized to the server to which you want to send the request. It has nothing to do with distributed search by itself.* But isn't the request sent to all the shards ? We set all the shard urls in the 'shards' parameter of our HttpRequest.Or is it something like the request is

eternal optimize interrupted

Hi, last evening we started an optimize over our solr index of 45GB. This morning the optimize was still running, discs spinning like crazy and de index directory has grew to 83GB. We stopped and restarted tomcat since solr was unresponsive and we needed to query the index. Now I don't know

Re: Rotating the primary shard in /solr/select

On Tue, Aug 4, 2009 at 2:37 PM, Rahul R rahul.s...@gmail.com wrote: *The SolrServer is initialized to the server to which you want to send the request. It has nothing to do with distributed search by itself.* But isn't the request sent to all the shards ? We set all the shard urls in the

Re: Picking Facet Fields by Frequency-in-Results

And further on this, if you want a field automatically added to each document with the list of its field names, check out http://issues.apache.org/jira/browse/SOLR-1280 Erik On Aug 4, 2009, at 1:01 AM, Avlesh Singh wrote: I understand the general need here. And just extending what

Re: Rotating the primary shard in /solr/select

Shalin, thank you for the clarification. Philip, I just realized that I have diverted the original topic of the thread. My apologies. Regards Rahul On Tue, Aug 4, 2009 at 3:35 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Tue, Aug 4, 2009 at 2:37 PM, Rahul R rahul.s...@gmail.com

Synonym aware string field typ

Hi all, I'd like to have a string type which is synonym aware at query time. Is it ok to have something like that: fieldType name=sastring class=solr.StrField analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.SynonymFilterFactory

Re: ClassCastException from custom request handler

Solr version: 1.3.0 694707 solrconfig.xml: requestHandler name=livecores class=LiveCoresHandler / public class LiveCoresHandler extends RequestHandlerBase { public void init(NamedList args) { } public String getDescription() { return ; } public String getSource() { return ; }

Re: ClassCastException from custom request handler

what is the package of LiveCoresHandler ? I guess the requestHandler name should be name=/livecores On Tue, Aug 4, 2009 at 5:04 PM, James Bradyjames.colin.br...@gmail.com wrote: Solr version: 1.3.0 694707 solrconfig.xml:    requestHandler name=livecores class=LiveCoresHandler / public class

Re: ClassCastException from custom request handler

Hi, the LiveCoresHandler is in the default package - the behaviour's the same if I have it in a properly namespaced package too... The requestHandler name can start either be a path (starting with '/') or a qt name: http://wiki.apache.org/solr/SolrRequestHandler 2009/8/4 Noble Paul നോബിള്‍

Solr 1.4 schedule?

Hi, When is Solr 1.4 scheduled for release? Is there any ballpark date yet? Thanks Rob

Delete solr data from disk space

I am facing a problem in deleting solr data form disk space. I had 80Gb of of solr data. I deleted 30% of these data by using query in solr-php client and committed. Now deleted data is not visible from the solr UI but used disk space is still 80Gb for solr data. Please reply if you have any

Re: Solr 1.4 schedule?

Very soon I think is the answer. As well as when its ready. Solr 1.4 is waiting for the next release of Lucene, which is very soon. Once Lucene comes out, Solr will follow in a week or two barring release issues. Also, if you look at JIRA:

Re: Delete solr data from disk space

Hello, A rigorous but quite effective method is manually deleting the files in your SOLR_HOME/data directory and reindex the documents you want. This will surely free some diskspace. Cheers, - Markus Jelsma Buyways B.V. Tel. 050-3118123 Technisch Architect

Re: Delete solr data from disk space

Sorry!! But this solution will not work because I deleted data by certain query. Then how can i know which files should be deleted. I cant delete whole data. -- View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808868.html Sent from the Solr -

Re: Delete solr data from disk space

Hi , Sorry!! But this solution will not work because I deleted data by certain query. Then how can i know which files should be deleted. I cant delete whole data. Markus Jelsma - Buyways B.V. wrote: Hello, A rigorous but quite effective method is manually deleting the files in your

Re: Delete solr data from disk space

You simply can't delete individual index files. Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: Ashish Kumar Srivastava ashu.impe...@gmail.com To:

Error with UpdateRequestProcessorFactory

Hi folks, I'm having some problem with a custom handler on my Solr. All the application works fine, but when I do a new checkout from svn and generate a jar file with my handler, I got: SEVERE: java.lang.NoSuchMethodError:

Re: Delete solr data from disk space

Hi Anish, Have you optimized your index? When you delete documents in lucene they are simply marked as 'deleted', they aren't physically removed from the disk. To get the disk space back you must run an optimize, which re-writes the index out to disk without the deleted documents,

Re: Synonym aware string field typ

Hi, KeywordTokenizer will not tokenize your string. I have a feeling that won't work with synonyms, unless your field value entirely match a synonym. Maybe an example would help: If you have: foo canine bar Then KeywordTokenizer won't break this into 3 tokens. And then canine/dog synonym

Re: Functions in search result

On Aug 4, 2009, at 4:37 AM, Markus Jelsma - Buyways B.V. wrote: Solr people, Can i retrieve results from a function query? For instance, i have a schema in which all documents have a size in bytes field. For each query, i also need to sum of the bytes field for the returned documents. I

Re: JVM Heap utilization Memory leaks with Solr

Hi Rahul, A) There are no known (to me) memory leaks. I think there are too many variables for a person to tell you what exactly is happening, plus you are dealing with the JVM here. :) Try jmap -histo:live PID-HERE | less and see what's using your memory. Otis -- Sematext is hiring --

Re: ClassCastException from custom request handler

There is *something* strange going on with classloaders; when I put my .class files in the right place in WEB-INF/lib in a repackaged solr.war file, it's not found by the plugin loader (Error loading class). So the plugin classloader isn't seeing stuff inside WEB-INF/lib. That explains why the

Re: Functions in search result

Markus, As far as I know, functions are executed on a per-document/field basis. That is, I don't think any of them aggregate numeric field values from a result set. Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER,

Re: 99.9% uptime requirement

On Mon, 3 Aug 2009 13:15:44 -0700 Robert Petersen rober...@buy.com wrote: Thanks all, I figured there would be more talk about daemontools if there were really a need. I appreciate the input and for starters we'll put two slaves behind a load balancer and grow it from there. Robert, not

Re: ClassCastException from custom request handler

Hi James! James Brady schrieb: There is *something* strange going on with classloaders; when I put my .class files in the right place in WEB-INF/lib in a repackaged solr.war file, it's not found by the plugin loader (Error loading class). So the plugin classloader isn't seeing stuff inside

Wild card search does not return any result

Hello All, I have two fields. field name=BUS type=text indexed=true stored=true/ field name=ROLE type=text indexed=true stored=true / I have document(which has been indexed) that has a value of ICS for BUS field and SSE for ROLE filed When I search for q=BUS:ics i get the result, but if

Re: Error with UpdateRequestProcessorFactory

On Tue, Aug 4, 2009 at 7:28 PM, Daniel Cassiano danielcassi...@gmail.comwrote: Hi folks, I'm having some problem with a custom handler on my Solr. All the application works fine, but when I do a new checkout from svn and generate a jar file with my handler, I got: SEVERE:

Re: ClassCastException from custom request handler

Hi Chantal! I've included a stack trace below. I've attached a debugger to the server starting up, and it is finding my class file as expected... I agree it looks like something wrong with how I've deployed the compiled code, but perhaps different Solr versions at compile time and run time?

Re: 99.9% uptime requirement

Right. You don't get to 99.9% by assuming that an 8 hour outage is OK. Design for continuous uptime, with plans for how long it takes to patch around a single point of failure. For example, if your load balancer is a single point of failure, make sure that you can redirect the front end

Re: Synonym aware string field typ

Hi Otis, Thanks. Yep, this synonym behaviour is the one I want. So if I don't want the synonyms to be applied at index time, I need to specify an index time analyzer right ? Jerome. 2009/8/4 Otis Gospodnetic otis_gospodne...@yahoo.com: Hi, KeywordTokenizer will not tokenize your string.

Re: ClassCastException from custom request handler

Hi there, could it be that something with the Generics code in the plugin loader classes works not as expected? Citing for example http://stackoverflow.com/questions/372250/java-generics-arrays-and-the-classcastexception this is because Generics only provide type-safety at compile-time.

Re: Wild card search does not return any result

Could it be the same reason as described here: http://markmail.org/message/ts65a6jok3ii6nva Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: Mohamed Parvez par...@gmail.com

Re: Synonym aware string field typ

Yes, you need to specify one or the other then, index-time or query-time, depending on where you want your synonyms to kick in. Eh, hitting reply to this email used your personal email instead of solr-user@lucene.apache.org . Eh eh. Making it hard for people replying to keep the discussion on

Re: ClassCastException from custom request handler

Code is from AbstractPluginLoader in the solr plugin package, 1.3 (the regular stable release, no svn checkout). 80-84 @SuppressWarnings(unchecked) protected T create( ResourceLoader loader, String name, String className, Node node ) throws Exception { return (T) loader.newInstance(

Re: Synonym aware string field typ

2009/8/4 Otis Gospodnetic otis_gospodne...@yahoo.com: Yes, you need to specify one or the other then, index-time or query-time, depending on where you want your synonyms to kick in. Ok great. Thx ! Eh, hitting reply to this email used your personal email instead of

Re: ClassCastException from custom request handler

Yeah I was thinking T would be SolrRequestHandler too. Eclipse's debugger can't tell me... Lot's of other handlers are created with no problem before my plugin falls over, so I don't think it's a problem with T not being what we expected. Do you know of any working examples of plugins I can

DisMax - fetching dynamic fields

Hi everybody, I have a couple of dynamic fields in my schema, e.g. rating_* popularity_* The problem I have is that if I try to specify existing fields rating_1 popularity_1 in fl parameter - DisMax handler just ignores them whereas StandardRequestHandler works fine. Any clues what's wrong?

Re: DisMax - fetching dynamic fields

Solr 1.4 built from trunk revision 790594 ( 02 Jul 2009 ) On Tue, Aug 4, 2009 at 9:19 PM, Alexey Serbaase...@gmail.com wrote: Hi everybody, I have a couple of dynamic fields in my schema, e.g. rating_* popularity_* The problem I have is that if I try to specify existing fields rating_1

Re: ClassCastException from custom request handler

James Brady schrieb: Yeah I was thinking T would be SolrRequestHandler too. Eclipse's debugger can't tell me... You could try disassembling. Or Eclipse opens classes in a very rudimentary format when there is no source code attached. Maybe it shows the actual return value there, instead of

Re: DIH: Any way to make update on db table?

Excellent, thanks Avlesh and Noble. -Jay On Mon, Aug 3, 2009 at 9:28 PM, Avlesh Singh avl...@gmail.com wrote: datasource.getData(update mytable ); //though the name is getData() it can execute update commands also Even when the dataSource is readOnly, Noble? Cheers Avlesh

Re: Wild card search does not return any result

Thanks Otis, The thread suggests that this is bug http://markmail.org/message/ts65a6jok3ii6nva#query:+page:1+mid:qinymqdn6mkocv4k Both SSE and ICS are 3 letter word and both are not part of English language. SEE* works fine and ICS* does not work, this is sure a bug. Any idea when will this bug

Dynamic Configuration

I have a client who is interested in using Solr/Lucene as their search engine. So far I think it meets 85% of their requirements. I have decided to integrate with JAMon tp provide statistical/performance analysis at run-time. The piece I am still missing is dynamic configuration of the indexing

Re: Error with UpdateRequestProcessorFactory

Hi Shalin, On Tue, Aug 4, 2009 at 12:43 PM, Shalin Shekhar Mangarshalinman...@gmail.com wrote: I'm having some problem with a custom handler on my Solr. All the application works fine, but when I do a new checkout from svn and generate a jar file with my handler, I got: SEVERE:

RE: facet sorting by index on sint fields

To solve this issue I created a subclass of SortableIntField that overrides the getSortField() method as follows... @Override public SortField getSortField(SchemaField field, boolean reverse) { return new SortField(field.getName(), SortField.INT, reverse);

Re: facet sorting by index on sint fields

On Thu, Jul 30, 2009 at 10:28 PM, Simon Stanlakesim...@tradebytes.com wrote: Hi, I have a field in my schema specified using field name=wordCount type=sint/ Where sint is specified as follows (the default from schema.xml) fieldType name=sint class=solr.SortableIntField sortMissingLast=true

Re: facet sorting by index on sint fields

On Tue, Aug 4, 2009 at 5:27 PM, Yonik Seeleyyo...@lucidimagination.com wrote: Is this a current limitation of solr faceting or am I missing a configuration step somewhere? I couldn't find any notes in the docs about this. This is not the intention - seems like a bug somewhere.  Is it still

RE: facet sorting by index on sint fields

Oh boy - I had a problem with my deploy scripts that was keeping an old version of the schema.xml file around. SortableIntField is working fine for me now. Sorry to waste everyone's time and thanks for the responses. Simon -Original Message- From: ysee...@gmail.com

Re: Dynamic Configuration

pgiesin wrote: I have a client who is interested in using Solr/Lucene as their search engine. So far I think it meets 85% of their requirements. I have decided to integrate with JAMon tp provide statistical/performance analysis at run-time. The piece I am still missing is dynamic configuration

Re: Wild card search does not return any result

Hi, I doubt it's a bug. It's probably working correctly based on the config, etc., I just don't have enough details about the configuration, your request handler, query rewriting, the data in your index, etc. to tell you what exactly is happening. Otis -- Sematext is hiring --

Re: eternal optimize interrupted

On Tue, Aug 4, 2009 at 6:04 AM, Thomas Kochtho...@koch.ro wrote: last evening we started an optimize over our solr index of 45GB. This morning the optimize was still running, discs spinning like crazy and de index directory has grew to 83GB. Hmmm, it was probably code to done given that

A Presentation on Building a Hadoop + Lucene System Architecture

Hey all, I just wanted to send a link to a presentation I made on how my company is building its entire core BI infrastructure around Hadoop, HBase, Lucene, and more. It features a decent amount of practical advice: from rules for approaching scalability problems, to why we chose certain aspects

Re: Wild card search does not return any result

You read it incorrectly Parvez. The bug that Bill seem to have found out is with the analysis tool and NOT the search handler itself. Results in your case is as expected. Wildcard queries are not analyzed hence the inconsistency. A workaround is suggested, on the same thread, here -

Re: JVM Heap utilization Memory leaks with Solr

Otis, Thank you for your response. I know there are a few variables here but the difference in memory utilization with and without shards somehow leads me to believe that the leak could be within Solr. I tried using a profiling tool - Yourkit. The trial version was free for 15 days. But I

Re: Dynamic Configuration

On Wed, Aug 5, 2009 at 12:59 AM, pgiesinpgie...@hubcitymedia.com wrote: I have a client who is interested in using Solr/Lucene as their search engine. So far I think it meets 85% of their requirements. I have decided to integrate with JAMon tp provide statistical/performance analysis at