Apologies - forgot to copy list.
On 15 March 2013 15:48, James Aley <[email protected]> wrote: > Hey Adrian, > > Thanks for the response. I was chatting to Sanne on IRC yesterday, and > he suggested this to me. Actually the logging I attached was from a > cluster of 4 servers with numOwners=2. Sorry, I should have mentioned > this actually, but I thought seeing as it didn't appear to make any > difference that I'd just keep things simple in my previous email. > > While it seemed not to make a difference in this case... I can see why > that would make sense. In future tests I guess I should probably stick > with numOwners > 1. > > > James. > > On 15 March 2013 15:44, Adrian Nistor <[email protected]> wrote: >> Hi James, >> >> I'm not an expert on InfinispanDirectory but I've noticed in [1] that the >> lucene-index cache is distributed with numOwners = 1. That means each cache >> entry is owned by just one cluster node and there's nowhere else to go in >> the cluster if the key is not available in local memory, thus it needs >> fetching from the cache store. This can be solved with numOwners > 1. >> Please let me know if this solves your problem. >> >> Cheers! >> >> >> On 03/15/2013 05:03 PM, James Aley wrote: >>> >>> Hey all, >>> >>> <OT> >>> Seeing as this is my first post, I wanted to just quickly thank you >>> all for Infinispan. So far I'm really enjoying working with it - great >>> product! >>> </OT> >>> >>> I'm using the InfinispanDirectory for a Lucene project at the moment. >>> We use Lucene directly to build a search product, which has high read >>> requirements and likely very large indexes. I'm hoping to make use of >>> a distribution mode cache to keep the whole index in memory across a >>> cluster of machines (the index will be too big for one server). >>> >>> The problem I'm having is that after loading a filesystem-based Lucene >>> directory into InfinispanDirectory via LuceneCacheLoader, no nodes are >>> retrieving data from the cluster - they instead look up keys in their >>> local CacheLoaders, which involves lots of disk I/O and is very slow. >>> I was hoping to just use the CacheLoader to initialize the caches, but >>> from there on read only from RAM (and network, of course). Is this >>> supported? Maybe I've misunderstood the purpose of the CacheLoader? >>> >>> To explain my observations in a little more detail: >>> * I start a cluster of two servers, using [1] as the cache config. >>> Both have a local copy of the Lucene index that will be loaded into >>> the InfinispanDirectory via the loader. This is a test configuration, >>> where I've set numOwners=1 so that I only need two servers for >>> distribution to happen. >>> * Upon startup, things look good. I see the memory usage of the JVM >>> reflect a pretty near 50/50 split of the data across both servers. >>> Logging indicates both servers are in the cluster view, all seems >>> fine. >>> * When I send a search query to either one of the nodes, I notice the >>> following: >>> - iotop shows huge (~100MB/s) disk I/O on that node alone from the >>> JVM process. >>> - no change in network activity between nodes (~300b/s, same as when >>> idle) >>> - memory usage on the node running the query increases dramatically, >>> and stays higher even after the query is finished. >>> >>> So it seemed to me like each node was favouring use of the CacheLoader >>> to retrieve keys that are not in memory, instead of using the cluster. >>> Does that seem reasonable? Is this the expected behaviour? >>> >>> I started to investigate this by turning on trace logging, in this >>> made me think perhaps the cause was that the CacheLoader's interceptor >>> is higher priority in the chain than the the distribution interceptor? >>> I'm not at all familiar with the design in any level of detail - just >>> what I picked up in the last 24 hours from browsing the code, so I >>> could easily be way off. I've attached the log snippets I thought >>> relevant in [2]. >>> >>> Any advice offered much appreciated. >>> Thanks! >>> >>> James. >>> >>> >>> [1] https://www.refheap.com/paste/12531 >>> [2] https://www.refheap.com/paste/12543 >>> _______________________________________________ >>> infinispan-dev mailing list >>> [email protected] >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
