map/reduce is used in nutch to do the actual processing of the index. the
searching (was when i last looked) a daemon which could run on mutliple
servers. looking at their solution for this is useful. I have done such a
thing myself before a while back.
a branch utilising boost-mpi exists for clucene, i haven't looked at it, but
as mpi is used for interproc i suppose it would be worth looking at
ben
On Thu, Sep 2, 2010 at 12:46 PM, suman holani <suman.hol...@gmail.com>wrote:
> Thanks Ben for quick reply.
>
>
>
> I read somewhere , Nutch uses mapper and reducer to do so...but really
> donno how to go abt it.
> On net I am not able to find gud papers related to it.
> If you could provide me the links where I can find more detail abt it.That
> would be helpful
>
> has parallel multi search engine being implemented in clucene....?
>
> Regards,
> Suman
>
>
>
> On Thu, Sep 2, 2010 at 1:12 PM, suman holani <suman.hol...@gmail.com>wrote:
>
>> Hello,
>>
>>
>>
>> I am using Clucene , and want to increase performance.
>>
>> does clucene something like parallel searching for distributed indexes.
>>
>> I want to reduce search time , which query takes for searching clucene
>> indexes.
>>
>>
>> 1. If I can I distribute my indexes such that each index , will have some
>> documents.
>> Now when I fire , query , multi searcher ll search the diff indexes in
>> *parallel
>> *and give me result .
>> [I just read :MultiSearcher` searches indices sequentially. /is there
>> nething else which I can use for that]
>>
>> Its jst like concept of using hadoop map reduce
>>
>>
>>
>> 2. how to do distributed indexing
>>
>>
>> 3. If I can make subset of docs in indexes , which only will be searched
>> in case of query is set.
>>
>> like Lets say I have hash map of
>>
>> P1 -1,2,3,4
>>
>> P2 - 3,4,5
>>
>> P3-7,5,3
>>
>>
>>
>> Now I have an documents in lucene index stored as
>>
>> 1-P1
>>
>> 2-P1
>>
>> 3-P1,P2,P3
>>
>> 4-P1,P2
>>
>> 5-P2,P3
>>
>> 7-P3
>>
>> ..
>>
>> ..
>>
>> when i search docs with P2 I get 3,4,5
>>
>> Now I want my search to b restricted to just 3,4,5 doc only. where by I
>> can search only these docs for further parameters.
>>
>>
>>
>> a. How to go abt it.
>>
>> b. Is there any other searching mechanism I should use, or CLucene is
>> better fit?
>>
>> c. should i keep my hash map also in lucene indexes and is then thr a
>> method to link it to another lucene indexes.
>>
>>
>>
>>
>>
>> Regards,
>> suman
>>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> CLucene-developers mailing list
> CLucene-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
CLucene-developers mailing list
CLucene-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/clucene-developers
