Re: Cassandra JVM configuration
On 2019-09-06 11:02, Oleksandr Shulgin wrote: On Fri, Sep 6, 2019 at 11:00 AM wrote: - reads => as much as possible - huge stream of requests - data => 186GB on each node - the reads are unpredictable - there's (in the cluster) about 6 billions of records I wonder though, if it makes sense to use Cassandra for a read-only dataset? Couldn't you just put it on something like Amazon S3 and be done with it? How many rows per partition do you have? Do you always scan full partition or you need to restrict results by clustering key? Regards, -- Alex Hi, well it might be updated, but updates can be done only after processing of data finished. No, accessing through the id. How can I get number of rows within the partition? Never need this. Thanks Pat Freehosting PIPNI - http://www.pipni.cz/ - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra JVM configuration
Pat, You might find this post useful in tuning G1 for Cassandra: * http://deliberate-thinking.blogspot.com/2019/05/tuning-g1-gc-for-cassandra.html This assumes a machine with 60-120G of RAM -- and your use case may be different than the clusters I've tuned, so take each step with care. Also, I have no experience tuning G1 over 31G of heap -- the JVM uses 8 byte pointers with a 32G+ heap. You can get a lot of read throughput with a 24G heap with a new size fixed at 8-10G, as long as your records aren't all huge blobs. John... From: p...@xvalheru.org Sent: Friday, September 6, 2019 3:00 AM To: user@cassandra.apache.org Cc: Jeff Jirsa Subject: Re: Cassandra JVM configuration - reads => as much as possible - huge stream of requests - data => 186GB on each node - the reads are unpredictable - there's (in the cluster) about 6 billions of records I'll try change the garbage collector. Thanks Pat On 2019-09-05 16:38, Jeff Jirsa wrote: > Lot of variables > > - how many reads per second per machine? > - how much data per machine? > - are the reads random or is there a hot working set? > > Some of the suggestions online are old. > CASSANDRA-8150 has some old’ish suggestions if you’re running CMS > collector. Running > 16G heap should consider using G1GC and that’s > tuned quite differently. Amy Tobey has a decent (2.1 era) tuning > guide, I imagine The Last Pickle has one as well (and wouldn’t be > surprised if Pythian and Instaclustr do too). > > > >> On Sep 5, 2019, at 7:04 AM, p...@xvalheru.org wrote: >> >> Hi, >> >> sorry to bring such question, but I want to ask what are the best JVM >> options for Cassandra node? In solution I'm implementing the Cassandra >> serves as read-only storage (of course populated at beginning) - the >> records are not changed in time. Currently each Cassandra node's VM >> has this configuration 16CPUs and 64GB of RAM. I've set these JVM >> options: -Xms4G and -Xmx40G; JDK 1.8.0_221. My question is what's the >> best size of memory for Cassandra node and if there's any relation >> between number of CPUs and memory size. When I've searched for an >> answer I've found that suggested size for node is 8GB of RAM, but I >> have doubts. >> >> Thanks >> >> Pat >> >> >> Freehosting PIPNI - >> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pipni.cz_&d=DwIDaQ&c=z0adcvxXWKG6LAMN6dVEqQ&r=W9UI0GQq10yOhf5LxSjoITGT9p69DtOfFK_UGgl4kx8&m=Twi1q-yZbM-7JseysmH7h_XO5CVH_qz-2rZkyrQ27zY&s=KB8dnJ9Hih2DBTUE5LHJNDQW5jpcUv9GPGwoC-03bmQ&e= >> >> >> - >> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: user-h...@cassandra.apache.org >> > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > > > Freehosting PIPNI - > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pipni.cz_&d=DwIDaQ&c=z0adcvxXWKG6LAMN6dVEqQ&r=W9UI0GQq10yOhf5LxSjoITGT9p69DtOfFK_UGgl4kx8&m=Twi1q-yZbM-7JseysmH7h_XO5CVH_qz-2rZkyrQ27zY&s=KB8dnJ9Hih2DBTUE5LHJNDQW5jpcUv9GPGwoC-03bmQ&e= Freehosting PIPNI - https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pipni.cz_&d=DwIDaQ&c=z0adcvxXWKG6LAMN6dVEqQ&r=W9UI0GQq10yOhf5LxSjoITGT9p69DtOfFK_UGgl4kx8&m=Twi1q-yZbM-7JseysmH7h_XO5CVH_qz-2rZkyrQ27zY&s=KB8dnJ9Hih2DBTUE5LHJNDQW5jpcUv9GPGwoC-03bmQ&e= - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra JVM configuration
On Fri, Sep 6, 2019 at 11:00 AM wrote: > - reads => as much as possible - huge stream of requests > - data => 186GB on each node > - the reads are unpredictable > - there's (in the cluster) about 6 billions of records > I wonder though, if it makes sense to use Cassandra for a read-only dataset? Couldn't you just put it on something like Amazon S3 and be done with it? How many rows per partition do you have? Do you always scan full partition or you need to restrict results by clustering key? Regards, -- Alex
Re: Cassandra JVM configuration
- reads => as much as possible - huge stream of requests - data => 186GB on each node - the reads are unpredictable - there's (in the cluster) about 6 billions of records I'll try change the garbage collector. Thanks Pat On 2019-09-05 16:38, Jeff Jirsa wrote: Lot of variables - how many reads per second per machine? - how much data per machine? - are the reads random or is there a hot working set? Some of the suggestions online are old. CASSANDRA-8150 has some old’ish suggestions if you’re running CMS collector. Running > 16G heap should consider using G1GC and that’s tuned quite differently. Amy Tobey has a decent (2.1 era) tuning guide, I imagine The Last Pickle has one as well (and wouldn’t be surprised if Pythian and Instaclustr do too). On Sep 5, 2019, at 7:04 AM, p...@xvalheru.org wrote: Hi, sorry to bring such question, but I want to ask what are the best JVM options for Cassandra node? In solution I'm implementing the Cassandra serves as read-only storage (of course populated at beginning) - the records are not changed in time. Currently each Cassandra node's VM has this configuration 16CPUs and 64GB of RAM. I've set these JVM options: -Xms4G and -Xmx40G; JDK 1.8.0_221. My question is what's the best size of memory for Cassandra node and if there's any relation between number of CPUs and memory size. When I've searched for an answer I've found that suggested size for node is 8GB of RAM, but I have doubts. Thanks Pat Freehosting PIPNI - http://www.pipni.cz/ - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org Freehosting PIPNI - http://www.pipni.cz/ Freehosting PIPNI - http://www.pipni.cz/ - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra JVM configuration
Thanks for suggestion. Pat On 2019-09-05 16:19, ZAIDI, ASAD wrote: Every use case is unique so as such jvm configs go with it. 8G may or may not be sufficient depending on live data you keep in, or fetch to memory. You can opt using G1GC, that is easy to start with. Some good suggestions are made if you want to try G1GC or stick with CMS. Take a look at [ https://tobert.github.io/pages/als-cassandra-21-tuning-guide.html ] -Original Message- From: p...@xvalheru.org [mailto:p...@xvalheru.org] Sent: Thursday, September 05, 2019 9:05 AM To: User Subject: Cassandra JVM configuration Hi, sorry to bring such question, but I want to ask what are the best JVM options for Cassandra node? In solution I'm implementing the Cassandra serves as read-only storage (of course populated at beginning) - the records are not changed in time. Currently each Cassandra node's VM has this configuration 16CPUs and 64GB of RAM. I've set these JVM options: -Xms4G and -Xmx40G; JDK 1.8.0_221. My question is what's the best size of memory for Cassandra node and if there's any relation between number of CPUs and memory size. When I've searched for an answer I've found that suggested size for node is 8GB of RAM, but I have doubts. Thanks Pat Freehosting PIPNI - https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pipni.cz_&d=DwICAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=FsmDztdsVuIKml8IDhdHdg&m=H2t7NfxXR_c8S_gVHKbkqoJYt2uJNvBvRcQBiEk2beI&s=OOXUbYzda5jQw9DZVUA4yExJv1gDNPzsxS0yqPlrDqY&e= - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org Freehosting PIPNI - http://www.pipni.cz/ Freehosting PIPNI - http://www.pipni.cz/ - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra JVM configuration
Lot of variables - how many reads per second per machine? - how much data per machine? - are the reads random or is there a hot working set? Some of the suggestions online are old. CASSANDRA-8150 has some old’ish suggestions if you’re running CMS collector. Running > 16G heap should consider using G1GC and that’s tuned quite differently. Amy Tobey has a decent (2.1 era) tuning guide, I imagine The Last Pickle has one as well (and wouldn’t be surprised if Pythian and Instaclustr do too). > On Sep 5, 2019, at 7:04 AM, p...@xvalheru.org wrote: > > Hi, > > sorry to bring such question, but I want to ask what are the best JVM options > for Cassandra node? In solution I'm implementing the Cassandra serves as > read-only storage (of course populated at beginning) - the records are not > changed in time. Currently each Cassandra node's VM has this configuration > 16CPUs and 64GB of RAM. I've set these JVM options: -Xms4G and -Xmx40G; JDK > 1.8.0_221. My question is what's the best size of memory for Cassandra node > and if there's any relation between number of CPUs and memory size. When I've > searched for an answer I've found that suggested size for node is 8GB of RAM, > but I have doubts. > > Thanks > > Pat > > > Freehosting PIPNI - http://www.pipni.cz/ > > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
RE: Cassandra JVM configuration
Every use case is unique so as such jvm configs go with it. 8G may or may not be sufficient depending on live data you keep in, or fetch to memory. You can opt using G1GC, that is easy to start with. Some good suggestions are made if you want to try G1GC or stick with CMS. Take a look at [ https://tobert.github.io/pages/als-cassandra-21-tuning-guide.html ] -Original Message- From: p...@xvalheru.org [mailto:p...@xvalheru.org] Sent: Thursday, September 05, 2019 9:05 AM To: User Subject: Cassandra JVM configuration Hi, sorry to bring such question, but I want to ask what are the best JVM options for Cassandra node? In solution I'm implementing the Cassandra serves as read-only storage (of course populated at beginning) - the records are not changed in time. Currently each Cassandra node's VM has this configuration 16CPUs and 64GB of RAM. I've set these JVM options: -Xms4G and -Xmx40G; JDK 1.8.0_221. My question is what's the best size of memory for Cassandra node and if there's any relation between number of CPUs and memory size. When I've searched for an answer I've found that suggested size for node is 8GB of RAM, but I have doubts. Thanks Pat Freehosting PIPNI - https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pipni.cz_&d=DwICAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=FsmDztdsVuIKml8IDhdHdg&m=H2t7NfxXR_c8S_gVHKbkqoJYt2uJNvBvRcQBiEk2beI&s=OOXUbYzda5jQw9DZVUA4yExJv1gDNPzsxS0yqPlrDqY&e= - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
