You could incorporate memcached or OSCache on frontend servers to try and cache as much as possible.
-byron --- Mike Alulin <[EMAIL PROTECTED]> wrote: > Thank you for your response. After reading this > document > http://wiki.apache.org/nutch/HardwareRequirements I > thought Nutch has its own cache. :o( > > RAMDirectory will help only in case one has enough > memory to store whole 100% of index files. It would > be better to have something like SQL Server has when > it caches in RAM only recently used pages. Although > I do not know much about the Nutch architecture and > have no idea whether any application specific > caching can be implemented. > > > Byron Miller <[EMAIL PROTECTED]> wrote: > Using that large of a JVM won't do anything but > slow > you down. I would warm up your index by throwing > queries at it to get the blocks cached on an OS > level > or work on implementing RAMDirectory instead of > FSDirectory to store your index in ram if you have > the > resources to do so. > > > > --- Mike Alulin wrote: > > > How to make Nutch use more RAM? I've added -Xmx3g > to > > the Tomcat command line, but Nutch does not seems > to > > use this memory. It does not go above 170MB. No > > matter whether I reserve 300MB or 3GB of RAM for > > JVM, search works with similar speed and requires > > similar amount of HDD reads. My test DB includes > > about 3M pages that according to the Nutch > hardware > > requirements article > > > (http://wiki.apache.org/nutch/HardwareRequirements) > > should take up to 6GB of RAM. How can I load more > > indexes to RAM? > > > > Test system info: > > OS: Windows Server 2K3 64bit > > RAM: 4GB > > JDK 1.5 64 bit > > Nutch DB ~3M pages > > > > > > > > --------------------------------- > > Yahoo! Photos > > Ring in the New Year with Photo Calendars. Add > > photos, events, holidays, whatever. > > > > > > --------------------------------- > Yahoo! Photos > Got holiday prints? See all the ways to get quality > prints in your hands ASAP.
