It's a long story... (TL;DR?)

2011-03-09 Thread Nelz
Hey all, Here's a long-ish story for you, and at the end I'll ask for some suggestions. We have been able to 'swimlane' our traffic into two realms, transactional (TX) and read-only (RO). This enables us to have lower load on our TX/master db, and have all the RO servers talk to replicated slave

Re: It's a long story... (TL;DR?)

2011-03-09 Thread dormando
#2 lead to a lot of talk of how to handle 'network fault tolerance'. I've realized that *most other* people's fault-tolerance is achieved by being on an intranet and the clients' ability to handle going from N - (N-1) physical nodes fairly gracefully. This doesn't really apply for us, because

Re: It's a long story... (TL;DR?)

2011-03-09 Thread Les Mikesell
On 3/9/11 8:36 PM, dormando wrote: So when you CRUD against the database, you send another command with the UDF in it to DELETE or SET against the memcached cluster local to that database. You can pair it inside a dummy INSERT or UPDATE or trigger or sproc or whatever floats your boat. (unless

Re: It's a long story... (TL;DR?)

2011-03-09 Thread dormando
On 3/9/11 8:36 PM, dormando wrote: So when you CRUD against the database, you send another command with the UDF in it to DELETE or SET against the memcached cluster local to that database. You can pair it inside a dummy INSERT or UPDATE or trigger or sproc or whatever floats your boat.

Re: It's a long story... (TL;DR?)

2011-03-09 Thread Adam Lee
i started to build somwthing similar, but rather than an in-app queue, it was external. basically, write your new entries into a fifo or a circular buffer on some external box(es) and have as many boxes as you need/can afford watching this and writing all entries to cache. this frees the app