Hi all,
How about adding hadoop support for distributed indexing. If required I
can start working on this. If Hadoop is the fesiable option.
Also what other technique one can think for doing distributed Indexing.
Currently I am planning on extending the SolrJ to keep a map of where
the document has gone and trying to get a distibuted Indexing.
--Thanks and Regards
Vaijanath
Otis Gospodnetic wrote:
That's right - most of them are about distributed searching (hence my notes
about sharding being up to the app). Hadoop's contrib/index is about dist
indexing:
"This contrib package provides a utility to build or update an index
using Map/Reduce.
A distributed "index" is partitioned into "shards". Each shard corresponds
to a Lucene instance."
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
From: Chris Hostetter <[EMAIL PROTECTED]>
To: java-user@lucene.apache.org
Sent: Monday, April 28, 2008 7:53:43 PM
Subject: Re: Does lucene support distributed indexing?
: There are actually several distributed indexing or searching projects in
: Lucene (the top-level ASF Lucene project, not Lucene Java), and it's
: time to start thinking about the possibility of bringing them together,
: finding commonalities, etc.
I would actually argue that almost all of the examples you listed describe
"distributed searching" to query multiple shards.
As far as i know, none of them address the "distributed indexing" aspect:
throw some raw data at the system and trust that it it will be indexed by
one (or more) shard(s) in a way that "evenly" distributes the indexing
"load"
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
