Re: unable to read saved rowcache from disk

2012-11-18 Thread aaron morton
Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: add a counter and print out myself On Thu, Nov 15, 2012 at 1:51 PM

Re: unable to read saved rowcache from disk

2012-11-18 Thread Manu Zhang
If you are using the off heap cache the upper bound is memory. If you are using the on head it's the JVM heap. But as I said earlier, I could not watch the usage of JVM heap while reading saved caches

Re: unable to read saved rowcache from disk

2012-11-16 Thread aaron morton
@cassandra.apache.org Sent: Thursday, November 15, 2012 12:15 PM Subject: Re: unable to read saved rowcache from disk For a row cache of 1,650,000: 16 byte token 300 byte row key ? and row data ? multiply by a java fudge factor or 5 or 10. Trying delete the saved cache and restarting. Cheers

Re: unable to read saved rowcache from disk

2012-11-15 Thread Edward Capriolo
smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: 3G, other jvm parameters are unchanged. On Thu, Nov 15, 2012 at 2:40 PM, Wz1975 wz1...@yahoo.com wrote: How big

Re: unable to read saved rowcache from disk

2012-11-15 Thread aaron morton
cache was not updated? Could be something else eats up the memory. You may profile memory and see who consumes the memory. Thanks. -Wei Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang

Re: unable to read saved rowcache from disk

2012-11-15 Thread Wei Zhu
...@thelastpickle.com To: user@cassandra.apache.org Sent: Thursday, November 15, 2012 12:15 PM Subject: Re: unable to read saved rowcache from disk For a row cache of 1,650,000: 16 byte token 300 byte row key ?  and row data ?  multiply by a java fudge factor or 5 or 10.  Trying delete the saved cache

Re: unable to read saved rowcache from disk

2012-11-14 Thread Wei Zhu
be more than 1M rows)    Thanks. -Wei From: Edward Capriolo edlinuxg...@gmail.com To: user@cassandra.apache.org Sent: Tuesday, November 13, 2012 11:13 PM Subject: Re: unable to read saved rowcache from disk http://wiki.apache.org/cassandra

Re: unable to read saved rowcache from disk

2012-11-14 Thread Manu Zhang
@cassandra.apache.org *Sent:* Tuesday, November 13, 2012 11:13 PM *Subject:* Re: unable to read saved rowcache from disk http://wiki.apache.org/cassandra/LargeDataSetConsiderations A negative side-effect of a large row-cache is start-up time. The periodic saving of the row cache information only

Re: unable to read saved rowcache from disk

2012-11-14 Thread Manu Zhang
) Thanks. -Wei -- *From:* Edward Capriolo edlinuxg...@gmail.com *To:* user@cassandra.apache.org *Sent:* Tuesday, November 13, 2012 11:13 PM *Subject:* Re: unable to read saved rowcache from disk http://wiki.apache.org/cassandra/LargeDataSetConsiderations

Re: unable to read saved rowcache from disk

2012-11-14 Thread Wz1975
Curious where did you see this?  Thanks. -Wei Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: OOM at deserializing 747321th row

Re: unable to read saved rowcache from disk

2012-11-14 Thread Manu Zhang
add a counter and print out myself On Thu, Nov 15, 2012 at 1:51 PM, Wz1975 wz1...@yahoo.com wrote: Curious where did you see this? Thanks. -Wei Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu

Re: unable to read saved rowcache from disk

2012-11-14 Thread Wz1975
How big is your heap?  Did you change the jvm parameter?  Thanks. -Wei Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: add a counter

Re: unable to read saved rowcache from disk

2012-11-14 Thread Manu Zhang
Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: OOM at deserializing 747321th row On Thu, Nov 15, 2012 at 9:08 AM, Manu Zhang owenzhang1...@gmail.comwrote: oh, as for the number

Re: unable to read saved rowcache from disk

2012-11-14 Thread Wz1975
.  You may profile memory and see who consumes the memory.  Thanks. -Wei Sent from my Samsung smartphone on ATT Original message Subject: Re: unable to read saved rowcache from disk From: Manu Zhang owenzhang1...@gmail.com To: user@cassandra.apache.org CC: 3G, other jvm

Re: unable to read saved rowcache from disk

2012-11-13 Thread Edward Capriolo
Yes the row cache could be incorrect so on startup cassandra verify they saved row cache by re reading. It takes a long time so do not save a big row cache. On Tuesday, November 13, 2012, Manu Zhang owenzhang1...@gmail.com wrote: I have a rowcache provieded by SerializingCacheProvider. The data

Re: unable to read saved rowcache from disk

2012-11-13 Thread Manu Zhang
incorrect... what do you mean? I think it's only 15MB, which is not big. On Wed, Nov 14, 2012 at 10:38 AM, Edward Capriolo edlinuxg...@gmail.comwrote: Yes the row cache could be incorrect so on startup cassandra verify they saved row cache by re reading. It takes a long time so do not save a

Re: unable to read saved rowcache from disk

2012-11-13 Thread Edward Capriolo
http://wiki.apache.org/cassandra/LargeDataSetConsiderations A negative side-effect of a large row-cache is start-up time. The periodic saving of the row cache information only saves the keys that are cached; the data has to be pre-fetched on start-up. On a large data set, this is probably going

Re: unable to read saved rowcache from disk

2012-11-13 Thread Manu Zhang
actually, I'm thinking about a bug or something On Wed, Nov 14, 2012 at 3:13 PM, Edward Capriolo edlinuxg...@gmail.comwrote: http://wiki.apache.org/cassandra/LargeDataSetConsiderations A negative side-effect of a large row-cache is start-up time. The periodic saving of the row cache