Re: customizable relevance engine

2010-08-03 Thread d' Ani
thanks DC for the boosting info.. can i plz have some code pointers to the same. As far as i understand the boosting is done during harvesting of the documents. Thats is fine but how do i achieve dynamic boosting...For example I wish to boost documents that have been opened more in the past (i a

Re: hit exception flushing segment _0 - IndexWriter configuration

2010-08-03 Thread Amin Mohammed-Coleman
Somewhat embarrassingly I can't seem to reproduce the problem anymore! I've tried to reproduce it for the last hour now and no luck. Sorry about that. If it happens again then I'll post back to the list. Thanks for your time. Amin On 3 Aug 2010, at 22:35, Michael McCandless wrote: > Can yo

RE: Question to the writer of MultiPassIndexSplitter

2010-08-03 Thread Christopher Condit
> I heard work is being done on re-writing MultiPassIndexSplitter so it will be > a > single pass and work quicker. Because that was so slow I just wrote a utility class to create a list of N IndexWriters and round robin documents to them as the index is created. Then we use a ParallelMultiSear

Re: Migrating from Lucene 2.9.1 to Solr 1.4.0 - Performance issues under heavy load

2010-08-03 Thread Lance Norskog
Is this an "apples to apples" comparison? That is, are you measuring the same complete flow on both apps? Does the Lucene app return fields via HTTP? On Tue, Aug 3, 2010 at 11:28 AM, Ophir Adiv wrote: > Hi, > > > > I’m currently involved in a project of migrating from Lucene 2.9.1 to Solr > 1.4.0

Re: hit exception flushing segment _0 - IndexWriter configuration

2010-08-03 Thread Michael McCandless
Can you post the full exception? And also the log output from IndexWriter.setInfoStream. Mike On Tue, Aug 3, 2010 at 5:28 PM, Amin Mohammed-Coleman wrote: > Hi > > Apologies for re sending this email but I was just wondering if any one might > be able to advise on the below. I'm not sure if I'

Re: hit exception flushing segment _0 - IndexWriter configuration

2010-08-03 Thread Amin Mohammed-Coleman
Hi Apologies for re sending this email but I was just wondering if any one might be able to advise on the below. I'm not sure if I've provided enough info. Again any help would be appreciated. Amin Sent from my iPhone On 1 Aug 2010, at 20:00, Amin Mohammed-Coleman wrote: > Hi > > I am cu

Re: get wordno, lineno, pageno for term/phrase

2010-08-03 Thread Erick Erickson
No, you can't do this with any existing analyzers I know of. Part of the problem here is that there's no good generic way to KNOW what a page and line are. Have you investigated payloads? I'm not sure that's a good fit for this particular problem, but it might be worth investigating. Best Erick

Re: Get fields from a Query object

2010-08-03 Thread Erick Erickson
H. Assuming you called rewrite, I'm going to have to defer that one, I'm not familiar enough with how range queries operate. But what version of Lucene are you using? Sorry I can't be more help Erick On Tue, Aug 3, 2010 at 5:02 AM, Anuj Shah wrote: > Thanks, that does seem good in theory.

Migrating from Lucene 2.9.1 to Solr 1.4.0 - Performance issues under heavy load

2010-08-03 Thread Ophir Adiv
Hi, I’m currently involved in a project of migrating from Lucene 2.9.1 to Solr 1.4.0. During stress testing, I encountered this performance problem: While actual search times in our shards (which are now running Solr) have not changed, the total time it takes for a query has increased dramatic

get wordno, lineno, pageno for term/phrase

2010-08-03 Thread arun r
hi all, I am new to Lucene. I am trying to use Lucene to generate data for a document classifier. I need to generate wordno, lineno, pageno for each term/phrase. I was able to use SpanQuery/SpanNearQuery to get the wordno (span.start()) for the term/phrase. To get pageno and lineno, a c

Re: customizable relevance engine

2010-08-03 Thread dc tech
Both Solr or Lucene allow extensive customization of relevance calculations. Examples include boosting matched in title field vs. Body, or boosting recent documents more than older documents. On 8/3/10, d' Ani wrote: > Hi all, > Is there any relevance engine that is built in lucence and which can

customizable relevance engine

2010-08-03 Thread d' Ani
Hi all, Is there any relevance engine that is built in lucence and which can be customized. Regards, Anirban De Yahoo: anirbande Skype: anirbande Gtalk : ade.sxc

Re: creating tag cloud (with faceted search?) for search result (filter)

2010-08-03 Thread dc tech
I assume that you are - building your own UI - and have some specific fields you want to show in a tag cloud? A generic tag cloud of all terms will not be very helpful as all the common words will dominate the tag cloud. What you want is to use some specific fields ie metadata for generating the t

Re: Get fields from a Query object

2010-08-03 Thread Anuj Shah
Thanks, that does seem good in theory. I can get the field from each of the terms and add them to a Set to de-dupe. However, in practice queries of the following nature seems to fail with an UnsupportedOperationException: field:a* field:[a TO b] Delving into the code a bit I see the following in