> On Feb 20, 2017, at 10:08 AM, Rob Archibald <r...@robarchibald.com> wrote: > > Ok, makes sense, but that’s a bummer, especially since the memcache library > in python already supports multi-server memcached > (http://stackoverflow.com/questions/4038094/using-multiple-memcache-servers-in-a-pool > > <http://stackoverflow.com/questions/4038094/using-multiple-memcache-servers-in-a-pool>). >
Hashing is a good and useful feature of some memcache clients, and is a feature of "python-memcached" which provides the "memcache" module seen in the stackoverflow post you linked. CalendarServer doesn't use this module, so we can't get that behavior for free or cheap. We kind of have our own memcache client - I think we even have multiple memcache implementations for historical (read: bad) reasons. What we have isn't ideal, but the features we lack in this area are easily implemented out-of-band. > Not sure yet, but I’ll probably just use > https://github.com/twitter/twemproxy <https://github.com/twitter/twemproxy> > which is a reverse proxy to actual memcached servers. It’s written by Twitter > and works well from what I’ve read. I tried using > https://github.com/zobo/mrproxy <https://github.com/zobo/mrproxy> which > proxies memcached protocol to Redis since I already had Redis servers but had > issues getting that working for some reason. I’ll report back when I get one > or the other working in case others are interested. Please do. Thanks, -dre > > Thanks! > Rob Archibald > > From: d...@apple.com [mailto:d...@apple.com] > Sent: Monday, February 20, 2017 9:46 AM > To: Rob Archibald > Cc: calendarserver-users@lists.macosforge.org > Subject: Re: [CalendarServer-users] Multiple Memcached servers > > Hi, > > CalendarServer doesn't have the ability to directly access more than one > memcache server per pool, and although you can bind "HandleCacheTypes" to > specific pools, that doesn't let you distribute data within a cachetype. > > To properly spread the load across multiple memcache servers, there needs to > be a strategy for mapping the keys to one of the memcache services, which is > already a feature of couchbase, and is known to work with CalendarServer. In > this setup, couchbase reverse proxies to other cousebase nodes for a non > local key, a behavior implemented both in their server and / or a local > process that has no data store and is just a proxy. > > -dre > > On Feb 18, 2017, at 7:18 PM, Rob Archibald <r...@robarchibald.com > <mailto:r...@robarchibald.com>> wrote: > > One of the benefits of Memcached is that it allows you to spread caching load > across multiple servers. From looking at the config file for Calendar Server, > it looked initially to me that it would support multiple Memcached servers, > but now I guess I’m not so sure. This is how php handles multi-server > configuration: > > $MEMCACHE_SERVERS = array( > "10.1.1.1", //web1 > "10.1.1.2", //web2 > "10.1.1.3", //web3 > ); > > I did a quick search of the code and I think I’ve found the connection method > for memcache. I found this in txdav/who/cache.py so let me know if I’ve gone > to the wrong place. It looks like it is only able to handle one server > (hightlighted below). Am I missing some secret method to allow for > multi-server memcached? > > def _getMemcacheClient(self, refresh=False): > """ > Get the memcache client instance to use for caching. > @param refresh: whether or not to create a new memcache client > @type refresh: L{bool} > @return: the client to use > @rtype: L{memcacheclient.Client} > """ > if refresh or not hasattr(self, "memcacheClient"): > if config.Memcached.Pools.Default.MemcacheSocket: > client_addr = > "unix:{}".format(config.Memcached.Pools.Default.MemcacheSocket) > else: > client_addr = "{}:{}".format( > config.Memcached.Pools.Default.BindAddress, > config.Memcached.Pools.Default.Port, > ) > self.memcacheClient = ClientFactory.getClient([client_addr], > debug=0, pickleProtocol=2) > return self.memcacheClient > > _______________________________________________ > calendarserver-users mailing list > calendarserver-users@lists.macosforge.org > <mailto:calendarserver-users@lists.macosforge.org> > https://lists.macosforge.org/mailman/listinfo/calendarserver-users > <https://lists.macosforge.org/mailman/listinfo/calendarserver-users>
_______________________________________________ calendarserver-users mailing list calendarserver-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-users
