storing in db 2.51020598412 for 24438 tweets storing in memcached 1.89508605003 reading from db 8.4804828167 reading from memcached 0.249507188797
Now this is awesome :D On Aug 18, 2:04 pm, Neeraj Agarwal <[email protected]> wrote: > storing in db 2.22120189667 for 24438 tweets > storing in memcached 1.91712522507 > reading from db 7.59235692024 (multi-get) > reading from memcached 1.85763192177 (single get) > > With libmemcached :) > > much improvement :D > > On Aug 18, 12:31 pm, dormando <[email protected]> wrote: > > > > > > > > > You should go read the manuals. > > > Look for "multiget" or mget or get_multi or whatever functions. I don't > > know what the equivalent set routines would look like for pylibmemc > > offhand. It should be in the documentation. > > > If not, and someone knows, they should pipe up :) > > > -Dormando > > > On Thu, 18 Aug 2011, Neeraj Agarwal wrote: > > > I have pasted my code above. > > > > I tried my program again by writing back the results in a file. > > > perhaps mongo just pushes the request and returns so it wasn't taking > > > much time while memcached is a blocking call. > > > > storing in db 1.26978802681 for 11437 tweets > > > storing in memcached 1.5911450386 > > > reading from db 41.3619318008 (was reading > > > one record at a time from db and writing it to the file) > > > reading from memcached 1.78016901016 (similarly with > > > memcached) > > > > mongodb still winning in terms of writing.. > > > > Does libmemcached groups multiple get/ set calls? Or are they even > > > blocking calls? > > > > Thanks for your time, > > > Neeraj > > > > On Aug 18, 12:08�pm, dormando <[email protected]> wrote: > > > > > 85 seconds was because of the network latency (was using EC2 with my > > > > > computer. pinging time was 350 ms itself..) > > > > > > Perhaps for x many number of items, it was taking x*350 ms time for > > > > > making calls.. while mongo, it was sending all data at one go. > > > > > > So I ran the script on the server itself: > > > > > storing in db �0.108298063278 �for �1487 �items > > > > > storing in memcached �0.208426952362 > > > > > reading from db �0.0738799571991 > > > > > reading from memcached �0.145488023758 > > > > > > what am I doing wrong here? > > > > > Can you attach your benchmark program? > > > > > You should be using multiget to fetch back items quicker. Try using > > > > pylibmc, which is based off of libmemcached. That may be able to do > > > > multisets and get you better speed. The other library is pure python, I > > > > think. Would be slower than a native DB driver. > > > > > -Dormando
