Chandraprakash: Thanks for the info! Great stuff, but it's lead to a few more questions...
> I had run a big mapred job (160GB data) on a small cluster of 7 nodes. I > had started 15 Memcached server instances on 7 nodes and I noticed that > a single memcached server was processing 1 million requests per second > (in my case), however it was definitely 3-4 times slower than in-memory > approach. I had to increase the limit of open file descriptors for that. Was the input for the mapred job coming from Tokyo Cabinet, or were you just writing the results of the mapred to TC? If you were using TC for input to mapred, how did you do the Input Splits? Did you write a custom splitter for Tokyo Cabinet? > Since memcached was not performing up to my expectations I used > Tokyocabinet ( A file based database) and its performance was near to > in-memory approach. Were you using Tokyo Cabinet over the network, that is, using Tokyo Tyrant? Or were you running and accessing a local TC process? Thanks for shedding some light on these additional questions... -- Andrzej Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com
