Re: Redis as Solr Cache
You touched an interesting point. I am really assuming if a quick win scenario is even possible. But what would be the advantage of using Redis to keep Solr Cache if each node would keep it's own Redis cache? 2013/12/29 Upayavira u...@odoko.co.uk On Sun, Dec 29, 2013, at 02:35 PM, Alexander Ramos Jardim wrote: While researching for Solr Caching options and interesting cases, I bumped on this https://github.com/dfdeshom/solr-redis-cache. Does anyone has any experience with this setup? Using Redis as Solr Cache. I see a lot of advantage in having a distributed cache for solr. One solr node benefiting from the cache generated on another one would be beautiful. I see problems too. Performance wise, I don't know if it would be viable for Solr to write it's cache through the network on Redis Master node. And what about if I have Solr nodes with different index version looking at the same cache? IMO as long as Redis is useful, if it isn't to have a distributed cache, I think it's not possible to get better performance using it. This idea makes assumptions about how a Solr/Lucene index operates. Certainly, in a SolrCloud setup, each node is responsible for its own committing, and its caches exist for the timespan between commits. Thus, the cache one node will need will not necessarily be the same as the one that is needed by another node, which might have a commit interval slightly out of sync with the first. So, whilst this may be possible, and may give some benefits, I'd reckon that it would be a rather substantial engineering exercise, rather than the quick win you seem to be assuming it might be. Upayavira -- Alexander Ramos Jardim
Re: Redis as Solr Cache
This is a neat idea, but could be too close to lucene/etc. You could jump up one level in the stack and use Redis/memcache as a distributed HTTP cache in conjunction with Solr's HTTP caching and a proxy. I tried doing this myself with Nginx, but I forgot what issue I hit - I think misses needed logic outside of nginx but I didn't spend too much time on it. Tim On 2 January 2014 07:51, Alexander Ramos Jardim alexander.ramos.jar...@gmail.com wrote: You touched an interesting point. I am really assuming if a quick win scenario is even possible. But what would be the advantage of using Redis to keep Solr Cache if each node would keep it's own Redis cache? 2013/12/29 Upayavira u...@odoko.co.uk On Sun, Dec 29, 2013, at 02:35 PM, Alexander Ramos Jardim wrote: While researching for Solr Caching options and interesting cases, I bumped on this https://github.com/dfdeshom/solr-redis-cache. Does anyone has any experience with this setup? Using Redis as Solr Cache. I see a lot of advantage in having a distributed cache for solr. One solr node benefiting from the cache generated on another one would be beautiful. I see problems too. Performance wise, I don't know if it would be viable for Solr to write it's cache through the network on Redis Master node. And what about if I have Solr nodes with different index version looking at the same cache? IMO as long as Redis is useful, if it isn't to have a distributed cache, I think it's not possible to get better performance using it. This idea makes assumptions about how a Solr/Lucene index operates. Certainly, in a SolrCloud setup, each node is responsible for its own committing, and its caches exist for the timespan between commits. Thus, the cache one node will need will not necessarily be the same as the one that is needed by another node, which might have a commit interval slightly out of sync with the first. So, whilst this may be possible, and may give some benefits, I'd reckon that it would be a rather substantial engineering exercise, rather than the quick win you seem to be assuming it might be. Upayavira -- Alexander Ramos Jardim
Redis as Solr Cache
While researching for Solr Caching options and interesting cases, I bumped on this https://github.com/dfdeshom/solr-redis-cache. Does anyone has any experience with this setup? Using Redis as Solr Cache. I see a lot of advantage in having a distributed cache for solr. One solr node benefiting from the cache generated on another one would be beautiful. I see problems too. Performance wise, I don't know if it would be viable for Solr to write it's cache through the network on Redis Master node. And what about if I have Solr nodes with different index version looking at the same cache? IMO as long as Redis is useful, if it isn't to have a distributed cache, I think it's not possible to get better performance using it. -- Alexander Ramos Jardim
Re: Redis as Solr Cache
On Sun, Dec 29, 2013, at 02:35 PM, Alexander Ramos Jardim wrote: While researching for Solr Caching options and interesting cases, I bumped on this https://github.com/dfdeshom/solr-redis-cache. Does anyone has any experience with this setup? Using Redis as Solr Cache. I see a lot of advantage in having a distributed cache for solr. One solr node benefiting from the cache generated on another one would be beautiful. I see problems too. Performance wise, I don't know if it would be viable for Solr to write it's cache through the network on Redis Master node. And what about if I have Solr nodes with different index version looking at the same cache? IMO as long as Redis is useful, if it isn't to have a distributed cache, I think it's not possible to get better performance using it. This idea makes assumptions about how a Solr/Lucene index operates. Certainly, in a SolrCloud setup, each node is responsible for its own committing, and its caches exist for the timespan between commits. Thus, the cache one node will need will not necessarily be the same as the one that is needed by another node, which might have a commit interval slightly out of sync with the first. So, whilst this may be possible, and may give some benefits, I'd reckon that it would be a rather substantial engineering exercise, rather than the quick win you seem to be assuming it might be. Upayavira
