Re: Indexing Tips and Hints

2003-02-25 Thread Andrzej Bialecki
Terry Steichen wrote: Hi Andrzej, Thanks for the code. I'll try it as soon as I have time. If you had a copy of the modified FSDirectory implementation you could also share, that would make testing it a bit quicker and easier. BTW, when you said it supposedly increases I/O, I gather that you

Re: Indexing Tips and Hints

2003-02-25 Thread Andrzej Bialecki
Terry Steichen wrote: Hi Andrzej, Thanks for the code. I'll try it as soon as I have time. If you had a copy of the modified FSDirectory implementation you could also share, that would make testing it a bit quicker and easier. Ask and ye shall receive... :-) Be aware, though, that I tested it

Re: Indexing Tips and Hints

2003-02-25 Thread Andrzej Bialecki
petite_abeille wrote: On Tuesday, Feb 25, 2003, at 09:43 Europe/Zurich, Andrzej Bialecki wrote: No, I'm not - this is clearly stated in the class javadoc. I meant to try it out in my application, but haven't got to it yet - I need to address first the base functionality, not performance; so, I

Re: Indexing Tips and Hints

2003-02-25 Thread petite_abeille
On Tuesday, Feb 25, 2003, at 11:48 Europe/Zurich, Andrzej Bialecki wrote: This is strange, or at least counter-intuitive - if you buffer larger parts of data in RAM than the standard implementation does, it should definitely be faster... Let's wait and see what Terry comes up with. BTW. how

Re: Indexing Tips and Hints

2003-02-25 Thread Michael Barry
Thanks for all the info. I've been working on streamlining my indexing and I've finally found the message from last year that intrigued me (http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]msgNo=1220). In that message, karl øie suggests 1. use a ramdir, and mutliple fsdirs 2. merge the

Indexing Tips and Hints

2003-02-24 Thread Michael Barry
All, I'm in need of some pointers, hints or tips on indexing large collections of data. I know I saw some tips on this list before but when I tried searching the list, I came up blank. I have a large collection of XML files (336000 files around 5K apiece) that I'm indexing and its taking

Re: Indexing Tips and Hints

2003-02-24 Thread Terry Steichen
List [EMAIL PROTECTED] Sent: Monday, February 24, 2003 2:00 PM Subject: Indexing Tips and Hints All, I'm in need of some pointers, hints or tips on indexing large collections of data. I know I saw some tips on this list before but when I tried searching the list, I came up blank. I have

Re: Indexing Tips and Hints

2003-02-24 Thread Andrzej Bialecki
[EMAIL PROTECTED] To: Lucene Users List [EMAIL PROTECTED] Sent: Monday, February 24, 2003 2:00 PM Subject: Indexing Tips and Hints All, I'm in need of some pointers, hints or tips on indexing large collections of data. I know I saw some tips on this list before but when I tried searching

Re: Indexing Tips and Hints

2003-02-24 Thread Otis Gospodnetic
to Onjava.com. It talks about the performance of indexing. I don't know when exactly it will be published, but when it does I'll send the URL to the list. Otis --- Michael Barry [EMAIL PROTECTED] wrote: All, I'm in need of some pointers, hints or tips on indexing large collections of data. I

Re: Indexing Tips and Hints

2003-02-24 Thread Terry Steichen
? Regards, Terry - Original Message - From: Andrzej Bialecki [EMAIL PROTECTED] To: Lucene Users List [EMAIL PROTECTED] Sent: Monday, February 24, 2003 3:59 PM Subject: Re: Indexing Tips and Hints Hello, Since you are trying this anyway, and looking for ways to improve indexing times