Re: [PROPOSAL] index server project

2006-11-21 Thread Bob Carpenter
Doug Cutting wrote: It seems that Nutch and Solr would benefit from a shared index serving infrastructure. ... An RPC mechanism would be used to communicate between nodes (probably Hadoop's). The system would be configured with a single master node that keeps track of where indexes are

Re: [PROPOSAL] index server project

2006-11-06 Thread Stefan Groschupf
Hi, do people think we are already in a stage where we can setup some basic infrastructure like mailing list and wiki and move the discussion to the new mailing list. Maybe setup a incubator project? I would be happy to help with such basic tasks. Stefan Am 31.10.2006 um 22:03 schrieb

Re: [PROPOSAL] index server project

2006-10-30 Thread Doug Cutting
Yonik Seeley wrote: On 10/18/06, Doug Cutting [EMAIL PROTECTED] wrote: We assume that, within an index, a file with a given name is written only once. Is this necessary, and will we need the lockless patch (that avoids renaming or rewriting *any* files), or is Lucene's current index behavior

Re: [Fwd: [PROPOSAL] index server project]

2006-10-20 Thread Otis Gospodnetic
That's distributed indexed, built on top of Sun Grid. The project won a $50K prize. - Original Message From: Alexandru Popescu [EMAIL PROTECTED] To: general@lucene.apache.org Sent: Thursday, October 19, 2006 10:19:00 AM Subject: Re: [Fwd: [PROPOSAL] index server project] I am

Re: [Fwd: [PROPOSAL] index server project]

2006-10-20 Thread Otis Gospodnetic
Original Message Subject: [PROPOSAL] index server project Date: Wed, 18 Oct 2006 14:17:30 -0700 From: Doug Cutting [EMAIL PROTECTED] Reply-To: general@lucene.apache.org To: general@lucene.apache.org It seems that Nutch and Solr would benefit from a shared index

Re: [Fwd: [PROPOSAL] index server project]

2006-10-19 Thread Alexandru Popescu
Message Subject: [PROPOSAL] index server project Date: Wed, 18 Oct 2006 14:17:30 -0700 From: Doug Cutting [EMAIL PROTECTED] Reply-To: general@lucene.apache.org To: general@lucene.apache.org It seems that Nutch and Solr would benefit from a shared index serving infrastructure. Other

RE: [PROPOSAL] index server project

2006-10-19 Thread Steven Parkes
I like the idea. I'm trying to figure out, in broad strokes, the overarching goals. Forgive me if this is obvious, I just want to be clear. The major goal is scale, right? A distributed server provides more oomph than a single-node server can. There are a number of dimensions in scale. You

Re: [PROPOSAL] index server project

2006-10-19 Thread Yonik Seeley
On 10/19/06, Steven Parkes [EMAIL PROTECTED] wrote: You mention partitioning of indexes, though mostly around delete. What about scalability of corpus size? Definitely in scope. Solr already has scalability of search volume via searchers behind of a load balancer all getting their index from

[PROPOSAL] index server project

2006-10-18 Thread Doug Cutting
It seems that Nutch and Solr would benefit from a shared index serving infrastructure. Other Lucene-based projects might also benefit from this. So perhaps we should start a new project to build such a thing. This could start either in java/contrib, or as a separate sub-project, depending on

Re: [PROPOSAL] index server project

2006-10-18 Thread Yonik Seeley
On 10/18/06, Doug Cutting [EMAIL PROTECTED] wrote: Does this make sense? Does it sound like it would be useful to Solr? To Nutch? To others? Who would be interested and able to work on it? Rather than holding my tounge until I wrap my head around all the issues, I'll say that I'm definitely