I have just provided you two options. Since you already store as part of the index, You could try external caching. Try using ehcache / Membase http://www.findbestopensource.com/tagged/distributed-caching . The caching system will do LRU and is much more efficient.
On Wed, Jul 14, 2010 at 12:39 PM, Li Li <fancye...@gmail.com> wrote: > I have already store it in lucene index. But it is in disk and When a > query come, it must seek the disk to get it. I am not familiar with > lucene cache. I just want to fully use my memory that load 10GB of it > in memory and a LRU stragety when cache full. To load more into > memory, I want to compress it "in memory". I don't care much about > disk space so whether or not it's compressed in lucene . > > 2010/7/14 findbestopensource <findbestopensou...@gmail.com>: > > You have two options > > 1. Store the compressed text as part of stored field in Solr. > > 2. Using external caching. > > http://www.findbestopensource.com/tagged/distributed-caching > > You could use ehcache / Memcache / Membase. > > > > The problem with external caching is you need to synchronize the > deletions > > and modification. Fetching the stored field from Solr is also faster. > > > > Regards > > Aditya > > www.findbestopensource.com > > > > > > On Wed, Jul 14, 2010 at 12:08 PM, Li Li <fancye...@gmail.com> wrote: > > > >> I want to cache full text into memory to improve performance. > >> Full text is only used to highlight in my application(But it's very > >> time consuming, My avg query time is about 250ms, I guess it will cost > >> about 50ms if I just get top 10 full text. Things get worse when get > >> more full text because in disk, it scatters erverywhere for a query.). > >> My full text per machine is about 200GB. The memory available for > >> store full text is about 10GB. So I want to compress it in memory. > >> Suppose compression ratio is 1:5, then I can load 1/4 full text in > >> memory. I need a Cache component for it. Has anyone faced the problem > >> before? I need some advice. Is it possbile using external tools such > >> as MemCached? Thank you. > >> > > >
