we use the fsdirectory, ramdirectory could be better on some edge cases, especially when you have a small index or do a lot of inserts and updates to the index. If you have enough memory on the machine, Lucene does a good job caching the termvectors and some relevant info, after some time the disk usage should be very low. The ramdirectory has a big penalty on load time has it have to load all the info in the memory, even that info that will never be used, that is a big cost with such a big index, also it doesn't ensure that it wouldn't be swapped to disk if the OS needs memory.
On Thu, Feb 26, 2009 at 6:07 PM, Glyn Darkin <g...@darkinsystems.com> wrote: > Thanks for the quick response Jokin, > > We are currently building a search solution that has a 3.6 g index > Max concurrent reads at the moment is 54, but we are hoping that this > will increase significantly as the website traffic increases > This will be a read only index, with a deployment of a new index nightly. > > On another note, > We are considering running searches against the index using a > FSDirectory. Is this how you have your search or do you load the index > into a RAMDirectory? > > Cheers > > Glyn > > > > > > > 2009/2/26 Jokin Cuadrado <joki...@gmail.com>: > > i have a 3 Gb index and another one of 800 Mb, the update rate is small > and > > the number or concurrent search right now it's also small, but i made the > > synchronization during a stress test without any problem, you just have > to > > copy the cfs file before the segments one, and as the most copy software > > list the directories in alphabetically order, it always happen. (cfs > files > > start with an "_" character, so they are always the first to be copied). > > in my case this works well because i don't have big restrictions, for > > example the result of the search could be diferent between 2 machines > during > > an small amount of time, and the incremental updates are done every > couple > > hours. > > > > If you tell us your constraints, I could suggest you more complex > approaches > > . > > > > > > > > > > On Thu, Feb 26, 2009 at 4:44 PM, Glyn Darkin <g...@darkinsystems.com> > wrote: > > > >> Jokin, > >> > >> What sort of index size are you dealing with and how many concurrent > >> searches would be running against your index when you Robocopy? > >> > >> Cheers > >> > >> Glyn > >> > >> > >> 2009/2/26 Jokin Cuadrado <joki...@gmail.com>: > >> > Robocopy /Mir works smoothly, just make sure that you copy first the > >> index > >> > files (.cfs) and after the segments.* ones. > >> > > >> > On Thu, Feb 26, 2009 at 5:35 AM, Nitin Shiralkar < > nit...@coreobjects.com > >> >wrote: > >> > > >> >> Hi All, > >> >> > >> >> Do we have any in-built replication support? Today, we are building > the > >> >> index and generating a periodic backup through the same builder > service. > >> >> This is working fine for couple of years. But I would like to know if > >> there > >> >> are any better options within Lucene library. Though we are using > >> >> Lucene.NET, but would also like to know if there is any support on > Java > >> side > >> >> if not on .NET. > >> >> > >> >> > >> >> Thanks & regards, > >> >> > >> >> Nitin Shiralkar > >> > > > > -- > > Jokin > > > > > > -- > Glyn Darkin > > Darkin Systems Ltd > Mob: 07961815649 > Fax: 08717145065 > Web: www.darkinsystems.com > > Company No: 6173001 > VAT No: 906350835 > -- Jokin