Re: Lucene indexing on Hadoop distributed file system

2006-03-27 Thread Doug Cutting
Igor Bolotin wrote: Does it make sense to change TermInfosWriter.FORMAT in the patch? Yes. This should be updated for any change to the format of the file, and this certainly constitutes a format change. This discussion should move to [EMAIL PROTECTED] Doug --

Re: Lucene indexing on Hadoop distributed file system

2006-03-27 Thread Igor Bolotin
Does it make sense to change TermInfosWriter.FORMAT in the patch? Igor On 3/27/06, Doug Cutting <[EMAIL PROTECTED]> wrote: > > Igor Bolotin wrote: > > If somebody is interested - I can post our changes in TermInfosWriter > and > > SegmentTermEnum code, although they are pretty trivial. > > Pleas

Re: Lucene indexing on Hadoop distributed file system

2006-03-27 Thread Andrzej Bialecki
Doug Cutting wrote: Igor Bolotin wrote: If somebody is interested - I can post our changes in TermInfosWriter and SegmentTermEnum code, although they are pretty trivial. Please submit this as a patch attached to a bug report. I contemplated making this change to Lucene myself, when writing

Re: Lucene indexing on Hadoop distributed file system

2006-03-27 Thread Doug Cutting
Igor Bolotin wrote: If somebody is interested - I can post our changes in TermInfosWriter and SegmentTermEnum code, although they are pretty trivial. Please submit this as a patch attached to a bug report. I contemplated making this change to Lucene myself, when writing Nutch's FsDirectory, b

Re: Lucene indexing on Hadoop distributed file system

2006-03-26 Thread Raghavendra Prabhu
I would like to see lucene operate with hadoop As you rightly pointed out, writing using FSDirectory to DFS would be a performance issue. I am interested in the idea. But i do not know how much time i can contribute to this because of the little time which i can spare. If anyone else is interest