Follow up my previous question. I tried to read those 6 million <key,value> records out. Both API are capable, but the spymemcache API is faster (16 minutes) than the Whalin's v2.5.1 distribution (24 minutes) (http://github.com/gwhalin/Memcached-Java-Client/downloads) .Since the spymemcache cannot insert that much, I came to a strange hybrid settings using Whalin's API to store, and using spymemcache to read ...
I am really curious why spy cannot store up to 6 million... On Sat, Oct 16, 2010 at 12:59 PM, Shi Yu <[email protected]> wrote: > Hi, > > I have two problems when using memcached java clients the spymemcached > (http://code.google.com/p/spymemcached/) and the gwhalin java client > for memcached (http://github.com/gwhalin/Memcached-Java-Client). I > found that the spymemcached failed to store more than 4.3 million > records sometimes 3.7 million (please see my code below). There was no > error no exception, but simply the code automatically stopped at the > 4.3 million and didn't even hit the final line. In contrast, the > gwhalin java client was able to insert 6 million records without > problem, however, comparing the speed of inserting the first 4 million > records that the gwhalin client is much slower than the spymemcached. > The memcached server is set up using the following command > "./memcached -d -m 4000 127.0.0.1 -p 11211" and I think there is no > problem at the server side. What is the problem here, should I adjust > any settings? Thanks? > > -Shi > > > //spymemcached code > public static void main(String[] args) throws Exception { > MemcachedClient mc=new MemcachedClient(new > InetSocketAddress("ocuic32.research", 11211)); > mc.flush(); > System.out.println("Memchaced flushed ..."); > int count = 0; > for(int i=0;i<6000000;i++){ > String a = "String"+i; > String b = "Value"+i; > > mc.add(a,i,(String) b); > count ++; > if (String.valueOf(count).endsWith("00000")) > System.out.println(count+ " elements added."); > } > > System.out.println("done "+ count +" records inserted"); > //spymemcached aint able to get this line > } > > > > //gwhalin memcached code > public static void main(String[] args) throws Exception { > BasicConfigurator.configure(); > String[] servers = { "ocuic32.research:11211" }; > SockIOPool pool = SockIOPool.getInstance(); > pool.setServers( servers ); > pool.setFailover( true ); > pool.setInitConn( 10 ); > pool.setMinConn( 5 ); > pool.setMaxConn( 250 ); > pool.setMaintSleep( 30 ); > pool.setNagle( false ); > pool.setSocketTO( 3000 ); > pool.setAliveCheck( true ); > pool.initialize(); > > MemcachedClient mcc = new MemcachedClient(); > mcc.flushAll(); > int count = 0; > int maxlength = 0; > //while((line=br.readLine())!=null){ > > for(int i=0;i<6000000;i++){ > String a = "String"+i; > String b = "Value"+i; > String sha1_ad1 = AeSimpleSHA1.SHA1(a); > mcc.set(sha1_ad1,(String) b); > count ++; > if (String.valueOf(count).endsWith("00000")) > System.out.println(count+ " elements added."); > > } > > System.out.println("done "+ count +" records > inserted"); //gwhalin 's client is able to get this line, but very > slow > } >
