Hi Dormando,
I got another issue with memcached. Once I am trying
with 14 GB some time, it able to pull 60 million data into memcached but
some time, my java client is keep thorwing this error and unable to set 60
million into memcached.
2012-08-01 11:54:29.255 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@5975d6ab has a ready op, handling IO
2012-08-01 11:54:29.255 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@5975d6ab has 1, interested in 5
2012-08-01 11:54:29.255 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@5975d6ab has a ready op, handling IO
2012-08-01 11:54:29.256 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@2993a66f has 1, interested in 5
2012-08-01 11:54:29.256 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@2993a66f has a ready op, handling IO
2012-08-01 11:54:29.256 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@2993a66f has 1, interested in 5
2012-08-01 11:54:29.256 INFO net.spy.memcached.MemcachedConnection:
sun.nio.ch.SelectionKeyImpl@2993a66f has a ready op, handling IO
2012-08-01 11:54:29.256 WARN net.spy.memcached.MemcachedConnection: Shut
down with 12462 bytes remaining to write
2012-08-01 11:54:29.257 INFO net.spy.memcached.MemcachedConnection: Shut
down memcached client
Please view stats detail of memcached
STAT libevent 1.4.8-stable
STAT pointer_size 64
STAT rusage_user 1395.867236
STAT rusage_system 179.303205
STAT curr_connections 5
STAT total_connections 60
STAT connection_structures 46
STAT reserved_fds 20
STAT cmd_get 100
STAT cmd_set 44122955
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 100
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 3759717871
STAT bytes_written 353029296
STAT limit_maxbytes 10485760000
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 1272597
STAT hash_power_level 25
STAT hash_bytes 268435456
STAT hash_is_expanding 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT bytes 6009690477
STAT curr_items 44122955
STAT total_items 44122955
STAT evictions 0
STAT reclaimed 0
END
On Monday, July 30, 2012 4:02:54 PM UTC+5:30, U.S. Adha wrote:
>
> Hi Dormando,
> Appreciated your help. Now am able to upload 60
> million as well.
>
>
> Thanks
> USA
>
>
> On Monday, July 30, 2012 2:13:56 PM UTC+5:30, Dormando wrote:
>>
>> bytes_written is the number of bytes written to the network.
>>
>> You're actually looking for the "bytes" value, which for you is about
>> 7.5G. The rest overhead from item headers and loss from slab class
>> offsets.
>>
>> On Mon, 30 Jul 2012, U.S. Adha wrote:
>>
>> > Yes, thats true, I over come with previous issue, but as stats detail
>> show, I have consume 5 GB(STAT bytes_written 512116959) and I have assign 8
>> GB (STAT limit_maxbytes 8388608000 ) to my memcached, so it seem to me, yet
>> I have 3 GB available. I have more RAM
>> > available, Let me try with 12 GB and let see if this would work..
>> >
>> >
>> > Thanks Dormando. I really appreciate your help.
>> >
>> >
>> >
>> > On Monday, July 30, 2012 1:40:08 PM UTC+5:30, Dormando wrote:
>> > Well if you have evictions, you're probably out of memory? This
>> isn't what
>> > you were saying before though.
>> >
>> > Get moar memoryyy?
>> >
>> > On Mon, 30 Jul 2012, U.S. Adha wrote:
>> >
>> > > Hi Dormando, Thanks for reply. Looks like
>> this works a bit, but it start evictions objects after 50 Millions object
>> inserted into memcached. I am using memcached 1.4.7 at this time. Also I
>> have tried with memcached 1.4.13 too, but same
>> > issue was
>> > > there too. Now am starting memcached with this
>> > >
>> > >
>> > > memcached -m 8000 -l 127.0.0.1 -p 11211 9000 9 >
>> log_nine_seconds
>> > >
>> > > yet my memcached evicting object...look stats
>> > >
>> > > STAT reclaimed 0
>> > > END
>> > > stats
>> > > STAT pid 5300
>> > > STAT uptime 3801
>> > > STAT time 1343635673
>> > > STAT version 1.4.7
>> > > STAT libevent 2.0.12-stable
>> > > STAT pointer_size 64
>> > > STAT rusage_user 500.823299
>> > > STAT rusage_system 486.058376
>> > > STAT curr_connections 5
>> > > STAT total_connections 37434
>> > > STAT connection_structures 27
>> > > STAT cmd_get 154795
>> > > STAT cmd_set 63614971
>> > > STAT cmd_flush 0
>> > > STAT get_hits 154795
>> > > STAT get_misses 0
>> > > STAT delete_misses 0
>> > > STAT delete_hits 0
>> > > STAT incr_misses 0
>> > > STAT incr_hits 0
>> > > STAT decr_misses 0
>> > > STAT decr_hits 0
>> > > STAT cas_misses 0
>> > > STAT cas_hits 0
>> > > STAT cas_badval 0
>> > > STAT auth_cmds 0
>> > > STAT auth_errors 0
>> > > STAT bytes_read 5421185438
>> > > STAT bytes_written 512116959
>> > > STAT limit_maxbytes 8388608000
>> > > STAT accepting_conns 1
>> > > STAT listen_disabled_num 0
>> > > STAT threads 4
>> > > STAT conn_yields 19532
>> > > STAT bytes 7523088744
>> > > STAT curr_items 55177103
>> > > STAT total_items 63614971
>> > > STAT evictions 8325645
>> > > STAT reclaimed 0
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Saturday, July 28, 2012 1:06:37 AM UTC+5:30, Dormando wrote:
>> > > Your startup options are bizarre...
>> > >
>> > > memcached -m 8000 -l 127.0.0.1 -p 11211
>> > > ^ just do that. don't add -L or -R and see if that helps
>> any.
>> > >
>> > > Also, we need to know the version and make of the client
>> you're using. Any
>> > > example code or data for how you're trying to load it
>> will be required to
>> > > see what's happening.
>> > >
>> > > Your client should be doing something weird to end up in
>> that state.
>> > > Memcached is trying to write back to your client but the
>> socket is closed
>> > > for some reason.
>> > >
>> > > On Fri, 27 Jul 2012, U.S. Adha wrote:
>> > >
>> > > > Yes, I tried without -vv option. It seem like, it's
>> speed increase a bit faster but not much as I expect. But this issue
>> > > > remain...................."Failed to write, and not due
>> to blocking: Connection reset by peer "
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Friday, July 27, 2012 3:05:59 PM UTC+5:30, U.S. Adha
>> wrote:
>> > > > Yes, I tried with -vv option. It seem like, it's
>> speed increase a bit faster but not much as I expect. But this issue
>> > > > remain...................."Failed to write, and
>> not due to blocking: Connection reset by peer "
>> > > >
>> > > >
>> > > >
>> > > > On Friday, July 27, 2012 3:03:48 PM UTC+5:30, Martin
>> Hellmich wrote:
>> > > > Have you tried the same without the -vv switch?
>> > > > I experienced memcached to be much faster without
>> the logging.
>> > > > Martin
>> > > >
>> > > > On 07/27/2012 11:29 AM, U.S. Adha wrote:
>> > > > > Hi, thanks for reply me. yet am fighting with
>> same issue, am trying to put data into memcached via java client and am
>> trying to
>> > > > put 60 million object into memcached. I am using
>> this command to start memcached
>> > > > >
>> > > > > memcached -d -vv -m 8000 -l 127.0.0.1 -p 11211
>> -c 250 -L 2048 -R 100 1000 10 > log_ten_seconds
>> > > > >
>> > > > > but after putting 20 million objects memcached
>> give me error like
>> > > > >
>> > > > > Failed to write, and not due to blocking:
>> Connection reset by peer
>> > > > > <80 connection closed.
>> > > > > Failed to write, and not due to blocking:
>> Connection reset by peer
>> > > > >
>> > > > >
>> > > > > Am not sure, where am having problem......I
>> really appreciate if you help me to get-out of this hall.
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wednesday, November 30, 2011 3:33:53 PM
>> UTC+5:30, ktechie wrote:
>> > > > >
>> > > > > My configuration is VMware on Red hat 64bit
>> linux OS. Memcached 1.4.5
>> > > > > and Xmemcached 1.3.5.
>> > > > > I have a program which does eager loading
>> of data, which is about 3GB,
>> > > > > this happens through a multithreaded
>> application, with 10 threads
>> > > > > I have also created a connection pool of 50
>> connections to memcached.
>> > > > > There is also a listener which tells when
>> there is disconnection or
>> > > > > the connection is healed.
>> > > > >
>> > > > > Sometimes the program works fine without
>> errors, but at times when I
>> > > > > see disconnections and connections
>> happening many times.
>> > > > > And the memcached server gives the
>> following errors.
>> > > > >
>> > > > > Failed to write, and not due to blocking:
>> Connection reset by peer
>> > > > > Failed to read, and not due to blocking:
>> > > > > errno: 104 Connection reset by peer
>> > > > > rcurr=127980b8 ritem=1442bd8a rbuf=12797c00
>> rlbytes=550 rsize=2048
>> > > > >
>> > > > > When this happens the data is not fully
>> loaded.
>> > > > >
>> > > > > At times this issue is resolved after the
>> VM is rebooted. but that is
>> > > > > not always the case.
>> > > > > I tried reducing the connection pool size
>> to 1 and thread pool to 1,
>> > > > > even then the disconnections could happen
>> > > > >
>> > > > > What could be the possible reasons for
>> this.
>> > > > >
>> > > >
>> > > > --
>> > > > Martin Hellmich Information
>> Technology Department
>> > > > [email protected]
>> CERN
>> > > > +41 22 76 765 26
>> CH-1211 Geneva 23
>> > > >
>> > > >
>> > > >
>> > >
>> > >
>> > >
>> >
>> >
>> >
>
>
