You do not need to include the segments when putting indexes in memory.
The distributed search makes two calls, the first for hits, the second
for hit details of top scoring pages.
Dennis
Alexander Aristov wrote:
Is it just the index size or including segments? You don't need segments in
RAm, only index files.
2008/8/12 Michael Chan <[EMAIL PROTECTED]>
Can a part of the index be loaded into RAM? For example, if the index is
20gb and I only have 8gb RAM, can I load 7gb of the index into RAM? Thanks.
Michael
On Tue, Aug 12, 2008 at 9:30 AM, Alexander Aristov <
[EMAIL PROTECTED]> wrote:
Look for a way to put your index into RAM. You create a file system which
works with RAM instead of hard disk and when copy your index into it.
It might significantly increas performance.
Alex
On 12/08/2008, Vijay <[EMAIL PROTECTED]> wrote:
Hi all,
I was wondering if you have any suggestions for fast serving
of queries with Nutch. I am running Nutch on a machine with 3GB
memory. The total size of my crawl directory is about 800MB. I was
wondering if there is any way to allow nutch to cache its indexes
either partly or wholly in main memory for faster serving of queries.
For some queries, especially after a period of "idleness" I often find
Nutch taking 7-8 seconds to return results for the query.
Likewise do let me know if there are ways to better utilize main
memory to speed up the indexing process.
Thanks,
Vijay
--
Best Regards
Alexander Aristov
