Hi Shi,
On 10/16/10 10:59 AM, Shi Yu wrote:
> Hi,
>
> I have two problems when using memcached java clients the spymemcached
> (http://code.google.com/p/spymemcached/) and the gwhalin java client
> for memcached (http://github.com/gwhalin/Memcached-Java-Client). I
> found that the spymemcached failed to store more than 4.3 million
> records sometimes 3.7 million (please see my code below). There was no
> error no exception, but simply the code automatically stopped at the
> 4.3 million and didn't even hit the final line. In contrast, the
> gwhalin java client was able to insert 6 million records without
> problem, however, comparing the speed of inserting the first 4 million
> records that the gwhalin client is much slower than the spymemcached.
> The memcached server is set up using the following command
> "./memcached -d -m 4000 127.0.0.1 -p 11211" and I think there is no
> problem at the server side. What is the problem here, should I adjust
> any settings? Thanks?
I can't think, off the top of my head, of any reason code in a tight
loop like that would stop. Try getting a stack dump after it's
"stopped"? Turn on some logging?
If it's with a freshly started memcached server, you can probably spot
from the curr_items stat how far it got. The other thing you may want
to be aware of is that the way memcached stores items and manages
memory, some of these items may be evicted to make space for new items.
Still, spy should run through completion.
- Matt
> -Shi
>
>
> //spymemcached code
> public static void main(String[] args) throws Exception {
> MemcachedClient mc=new MemcachedClient(new
> InetSocketAddress("ocuic32.research", 11211));
> mc.flush();
> System.out.println("Memchaced flushed ...");
> int count = 0;
> for(int i=0;i<6000000;i++){
> String a = "String"+i;
> String b = "Value"+i;
>
> mc.add(a,i,(String) b);
> count ++;
> if (String.valueOf(count).endsWith("00000"))
> System.out.println(count+ " elements added.");
> }
>
> System.out.println("done "+ count +" records inserted");
> //spymemcached aint able to get this line
> }
>
>
>
> //gwhalin memcached code
> public static void main(String[] args) throws Exception {
> BasicConfigurator.configure();
> String[] servers = { "ocuic32.research:11211" };
> SockIOPool pool = SockIOPool.getInstance();
> pool.setServers( servers );
> pool.setFailover( true );
> pool.setInitConn( 10 );
> pool.setMinConn( 5 );
> pool.setMaxConn( 250 );
> pool.setMaintSleep( 30 );
> pool.setNagle( false );
> pool.setSocketTO( 3000 );
> pool.setAliveCheck( true );
> pool.initialize();
>
> MemcachedClient mcc = new MemcachedClient();
> mcc.flushAll();
> int count = 0;
> int maxlength = 0;
> //while((line=br.readLine())!=null){
>
> for(int i=0;i<6000000;i++){
> String a = "String"+i;
> String b = "Value"+i;
> String sha1_ad1 = AeSimpleSHA1.SHA1(a);
> mcc.set(sha1_ad1,(String) b);
> count ++;
> if (String.valueOf(count).endsWith("00000"))
> System.out.println(count+ " elements added.");
>
> }
>
> System.out.println("done "+ count +" records
> inserted"); //gwhalin 's client is able to get this line, but very
> slow
> }
