Hi all, Thanks a lot guys for all the help.
Thanks y1rm3y4hu On 08-Feb-2011, at 12:53 PM, dormando wrote: > s/leaks/leads to > > On Mon, 7 Feb 2011, dormando wrote: > >> Well you were saying speed is the point, but RAM is there as well. >> >> If you properly tune the TCP stack the memory usage isn't bad at all. I've >> ran a number of hosts with 100,000+ tcp connections on them at once and >> while RAM gets sorta heavy it doesn't implode or anything. I say things >> based on practical experience running huge shit; memcached was designed >> and has been further tuned to run with 10,000+ connections just fine. >> >> UDP still uses buffers, but the requests disappear when you overflow, >> which leaks to the memcache client needing to wait on a timeout to ensure >> its response is really gone. In TCP land (ignoring the first SYN/ACK >> sequence) it can drop and retry packets with a relatively short timeout. >> Meaning you get the answer back and things work fine. >> >> There *are* some cases where UDP can be useful, but we would absolutely >> never recommend anyone do that unless they have to. The point is sort of >> moot since most people use data too large for memcached to handle. Those >> who've wanted UDP to handle larger packets end up reimplementing TCP on >> top of UDP to make it work. >> >> So, we've opted to not do that. >> >> On Tue, 8 Feb 2011, Roberto Spadim wrote: >> >>> not about faster or not, the point of udp is the ram used to allow tcp >>> connection alive, udp don't need ram to allow connections (just server >>> side, or when send/receive package), it's connectionless... (you know, >>> i know, everyone that use it know) >>> with many clients (more than 10000) udp for my benchmarks works better >>> than tcp (read more before above) >>> i know about limitations of udp protocol and lower layers (packet >>> fragmentation and others problems), it's good for some type of values >>> (data size) and network layout (internet / intranet) >>> for example in a local network it's very good, for a internet with >>> many routers a tcp connection is better, the point is, what type of >>> value is being stored, and what's the network layout? a big value >>> (more than 1kbyte) or a small? if all values are small, udp works very >>> well on local network, i use it without bugs... with jumbo frame you >>> can get more than 1kbyte without data loss with udp >>> >>> speed isn't the point, the data size the network layout and the cache >>> hit rate is the point, with broken packet we have no communication =] >>> >>> 2011/2/8 dormando <[email protected]>: >>>> Hi, >>>> >>>> I don't want to be rude but can you perhaps stop advocating using UDP? >>>> It's not actually faster if using persistent connections and is full of >>>> bugs and limitations (like a max packet size of 1.4k). >>>> >>>> Uhm. Actually in general your information is a little off from how we >>>> usually go about things; perhaps you could read some of the history or pad >>>> through the wiki a bit? I much enjoy your enthusiasm but there're good >>>> reasons why we recommend a list of other things for people to try first. >>>> >>>> ie; striping/replicating data halves your effective cache size and can >>>> introduce bugs. General you benefit more from performance by using more >>>> RAM. >>>> >>>> "UDP is faster than TCP" is ... mild failure as general knowledge. It's >>>> more complicated than that :( I'm glad more online games are moving away >>>> from UDP and toward TCP connections, as NAT'ing UDP is buggy and >>>> slaughtering slow connections with extra traffic wastes bandwidth for >>>> everyone involved. >>>> >>>> On Tue, 8 Feb 2011, Roberto Spadim wrote: >>>> >>>>> 1) some libraries implement hash to stripe informations (like raid0 do >>>>> with disks), you should use deterministic hash function (always set >>>>> the key, to the same server) >>>>> 2) failover should be a mirror flag (like raid1 with disks), it should >>>>> write to all servers that variable (write on all servers = write and >>>>> wait all servers to talk: that's ok), in case of a server problem, all >>>>> servers have the same information (you can use repcache, a memcache >>>>> similar server, with same memcache protocol and based on memcache, but >>>>> with replication feature, in this case replication is done in server, >>>>> not in client, check if it's a good sync time for you, and if it's a >>>>> network problem or not) >>>>> 3) no, you can use UDP in a good network, it's faster (don't need >>>>> connection) and don't have a lot of latency (TCP can have latency, but >>>>> some options can reduce it) persistent connection remove the >>>>> connection time, but it's make another problem... the TCP list get >>>>> bigger, maybe you TCP list can overflow the operational system TCP >>>>> list, and some connections must be closed... UDP don't have this >>>>> problem, it's connectionless =) >>>>> >>>>> 2011/2/8 y1rm3y4hu <[email protected]>: >>>>>> Hi, >>>>>> >>>>>> I've been trying to find resources online to address a few questions i >>>>>> had regarding the various configuration options available with >>>>>> Memcached client/server without much success. >>>>>> >>>>>> Heres how my setup would look like >>>>>> i'd have two web servers [amazon EC2 instances] load balancing >>>>>> incoming requests in a round robin fashion - each of these web servers >>>>>> would have memcached[client and server] installed in it >>>>>> >>>>>> Now it would be great if somebody could give me pointers on the below >>>>>> questions. >>>>>> >>>>>> #1) Should i use consistent hashing. >>>>>> I am not expecting instances to go down randomly. But whenever one >>>>>> machine has to be taken out for maintenance etc, would like to >>>>>> minimize the impact. i read about a reduced performance when switched >>>>>> to consistent hashing. Not sure whether it is still valid. >>>>>> >>>>>> #2 ) If we are using standard vs consistent hashing how would failover >>>>>> work? >>>>>> I see that pecl/memcache has a failover flag but can't find anything >>>>>> similar to it in pecl/memcached. What are the implications. >>>>>> >>>>>> #3) Should i always go with persistent connections? >>>>>> >>>>>> >>>>>> Any help/links/pointers would be highly appreciated :) >>>>>> >>>>>> >>>>>> Have a good day >>>>>> y1rm3y4hu >>>>> >>>>> >>>>> >>>>> -- >>>>> Roberto Spadim >>>>> Spadim Technology / SPAEmpresarial >>>>> >>>> >>> >>> >>> >>> -- >>> Roberto Spadim >>> Spadim Technology / SPAEmpresarial >>> >>
