Re: Cassandra 1.x and proper JNA setup
Super - thank you for help :) On Thu, Nov 3, 2011 at 6:55 PM, Jonathan Ellis wrote: > Relying on that was always a terrible idea because you could easily > OOM before it could help. There's no substitute for "don't make the > caches too large" in the first place. > > We're working on https://issues.apache.org/jira/browse/CASSANDRA-3143 > to make cache sizing easier. > > On Thu, Nov 3, 2011 at 3:16 AM, Maciej Miklas > wrote: > > According to source code, JNA is being used to call malloc and free. In > this > > case each cached row will be serialized into RAM. > > We must be really careful when defining cache size - to large size would > > cause out of memory. Previous Cassandra releases has logic that would > > decrease cache size if heap is low. > > Currently each row will be serialized without any memory limit checks - > > assuming that I understood it right. > > > > Those properties: > >reduce_cache_sizes_at: 0.85 > >reduce_cache_capacity_to: 0.6 > > are not used anymore - at least not when JNA is enabled, witch is default > > from Cassandra 1.0 > > > > > > On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas > > > wrote: > >> > >> I've just found, that JNA will be not used from 1.1 release - > >> https://issues.apache.org/jira/browse/CASSANDRA-3271 > >> I would be also nice to know what was the reason for this decision. > >> > >> Regards, > >> Maciej > >> > >> On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov > >> wrote: > >>> > >>> Up, also interested in answers to questions below. > >>> > >>> > >>> Best regards/ Pagarbiai > >>> > >>> Viktor Jevdokimov > >>> Senior Developer > >>> > >>> Email: viktor.jevdoki...@adform.com > >>> Phone: +370 5 212 3063 > >>> Fax: +370 5 261 0453 > >>> > >>> J. Jasinskio 16C, > >>> LT-01112 Vilnius, > >>> Lithuania > >>> > >>> > >>> > >>> Disclaimer: The information contained in this message and attachments > is > >>> intended solely for the attention and use of the named addressee and > may be > >>> confidential. If you are not the intended recipient, you are reminded > that > >>> the information remains the property of the sender. You must not use, > >>> disclose, distribute, copy, print or rely on this e-mail. If you have > >>> received this message in error, please contact the sender immediately > and > >>> irrevocably delete this message and any copies.-Original > Message- > >>> From: Maciej Miklas [mailto:mac.mik...@googlemail.com] > >>> Sent: Tuesday, November 01, 2011 11:15 > >>> To: user@cassandra.apache.org > >>> Subject: Cassandra 1.x and proper JNA setup > >>> > >>> Hi all, > >>> > >>> is there any documentation about proper JNA configuration? > >>> > >>> I do not understand few things: > >>> > >>> 1) Does JNA use JVM heap settings? > >>> > >>> 2) Do I need to decrease max heap size while using JNA? > >>> > >>> 3) How do I limit RAM allocated by JNA? > >>> > >>> 4) Where can I see / monitor row cache size? > >>> > >>> 5) I've configured JNA just for test on my dev computer and so far I've > >>> noticed serious performance issues (high cpu usage on heavy write > load), so > >>> I must be doing something wrong I've just copied JNA jars into > >>> Cassandra/lib, without installing any native libs. This should not > work at > >>> all, right? > >>> > >>> Thanks, > >>> Maciej > >>> > >> > > > > > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of DataStax, the source for professional Cassandra support > http://www.datastax.com >
Re: Cassandra 1.x and proper JNA setup
Relying on that was always a terrible idea because you could easily OOM before it could help. There's no substitute for "don't make the caches too large" in the first place. We're working on https://issues.apache.org/jira/browse/CASSANDRA-3143 to make cache sizing easier. On Thu, Nov 3, 2011 at 3:16 AM, Maciej Miklas wrote: > According to source code, JNA is being used to call malloc and free. In this > case each cached row will be serialized into RAM. > We must be really careful when defining cache size - to large size would > cause out of memory. Previous Cassandra releases has logic that would > decrease cache size if heap is low. > Currently each row will be serialized without any memory limit checks - > assuming that I understood it right. > > Those properties: > reduce_cache_sizes_at: 0.85 > reduce_cache_capacity_to: 0.6 > are not used anymore - at least not when JNA is enabled, witch is default > from Cassandra 1.0 > > > On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas > wrote: >> >> I've just found, that JNA will be not used from 1.1 release - >> https://issues.apache.org/jira/browse/CASSANDRA-3271 >> I would be also nice to know what was the reason for this decision. >> >> Regards, >> Maciej >> >> On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov >> wrote: >>> >>> Up, also interested in answers to questions below. >>> >>> >>> Best regards/ Pagarbiai >>> >>> Viktor Jevdokimov >>> Senior Developer >>> >>> Email: viktor.jevdoki...@adform.com >>> Phone: +370 5 212 3063 >>> Fax: +370 5 261 0453 >>> >>> J. Jasinskio 16C, >>> LT-01112 Vilnius, >>> Lithuania >>> >>> >>> >>> Disclaimer: The information contained in this message and attachments is >>> intended solely for the attention and use of the named addressee and may be >>> confidential. If you are not the intended recipient, you are reminded that >>> the information remains the property of the sender. You must not use, >>> disclose, distribute, copy, print or rely on this e-mail. If you have >>> received this message in error, please contact the sender immediately and >>> irrevocably delete this message and any copies.-Original Message- >>> From: Maciej Miklas [mailto:mac.mik...@googlemail.com] >>> Sent: Tuesday, November 01, 2011 11:15 >>> To: user@cassandra.apache.org >>> Subject: Cassandra 1.x and proper JNA setup >>> >>> Hi all, >>> >>> is there any documentation about proper JNA configuration? >>> >>> I do not understand few things: >>> >>> 1) Does JNA use JVM heap settings? >>> >>> 2) Do I need to decrease max heap size while using JNA? >>> >>> 3) How do I limit RAM allocated by JNA? >>> >>> 4) Where can I see / monitor row cache size? >>> >>> 5) I've configured JNA just for test on my dev computer and so far I've >>> noticed serious performance issues (high cpu usage on heavy write load), so >>> I must be doing something wrong I've just copied JNA jars into >>> Cassandra/lib, without installing any native libs. This should not work at >>> all, right? >>> >>> Thanks, >>> Maciej >>> >> > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Cassandra 1.x and proper JNA setup
According to source code, JNA is being used to call malloc and free. In this case each cached row will be serialized into RAM. We must be really careful when defining cache size - to large size would cause out of memory. Previous Cassandra releases has logic that would decrease cache size if heap is low. Currently each row will be serialized without any memory limit checks - assuming that I understood it right. Those properties: reduce_cache_sizes_at: 0.85 reduce_cache_capacity_to: 0.6 are not used anymore - at least not when JNA is enabled, witch is default from Cassandra 1.0 On Wed, Nov 2, 2011 at 1:53 PM, Maciej Miklas wrote: > I've just found, that JNA will be not used from 1.1 release - > https://issues.apache.org/jira/browse/CASSANDRA-3271 > I would be also nice to know what was the reason for this decision. > > Regards, > Maciej > > > On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov < > viktor.jevdoki...@adform.com> wrote: > >> Up, also interested in answers to questions below. >> >> >> Best regards/ Pagarbiai >> >> Viktor Jevdokimov >> Senior Developer >> >> Email: viktor.jevdoki...@adform.com >> Phone: +370 5 212 3063 >> Fax: +370 5 261 0453 >> >> J. Jasinskio 16C, >> LT-01112 Vilnius, >> Lithuania >> >> >> >> Disclaimer: The information contained in this message and attachments is >> intended solely for the attention and use of the named addressee and may be >> confidential. If you are not the intended recipient, you are reminded that >> the information remains the property of the sender. You must not use, >> disclose, distribute, copy, print or rely on this e-mail. If you have >> received this message in error, please contact the sender immediately and >> irrevocably delete this message and any copies.-Original Message- >> From: Maciej Miklas [mailto:mac.mik...@googlemail.com] >> Sent: Tuesday, November 01, 2011 11:15 >> To: user@cassandra.apache.org >> Subject: Cassandra 1.x and proper JNA setup >> >> Hi all, >> >> is there any documentation about proper JNA configuration? >> >> I do not understand few things: >> >> 1) Does JNA use JVM heap settings? >> >> 2) Do I need to decrease max heap size while using JNA? >> >> 3) How do I limit RAM allocated by JNA? >> >> 4) Where can I see / monitor row cache size? >> >> 5) I've configured JNA just for test on my dev computer and so far I've >> noticed serious performance issues (high cpu usage on heavy write load), so >> I must be doing something wrong I've just copied JNA jars into >> Cassandra/lib, without installing any native libs. This should not work at >> all, right? >> >> Thanks, >> Maciej >> >> >
Re: Cassandra 1.x and proper JNA setup
I've just found, that JNA will be not used from 1.1 release - https://issues.apache.org/jira/browse/CASSANDRA-3271 I would be also nice to know what was the reason for this decision. Regards, Maciej On Wed, Nov 2, 2011 at 1:34 PM, Viktor Jevdokimov < viktor.jevdoki...@adform.com> wrote: > Up, also interested in answers to questions below. > > > Best regards/ Pagarbiai > > Viktor Jevdokimov > Senior Developer > > Email: viktor.jevdoki...@adform.com > Phone: +370 5 212 3063 > Fax: +370 5 261 0453 > > J. Jasinskio 16C, > LT-01112 Vilnius, > Lithuania > > > > Disclaimer: The information contained in this message and attachments is > intended solely for the attention and use of the named addressee and may be > confidential. If you are not the intended recipient, you are reminded that > the information remains the property of the sender. You must not use, > disclose, distribute, copy, print or rely on this e-mail. If you have > received this message in error, please contact the sender immediately and > irrevocably delete this message and any copies.-Original Message- > From: Maciej Miklas [mailto:mac.mik...@googlemail.com] > Sent: Tuesday, November 01, 2011 11:15 > To: user@cassandra.apache.org > Subject: Cassandra 1.x and proper JNA setup > > Hi all, > > is there any documentation about proper JNA configuration? > > I do not understand few things: > > 1) Does JNA use JVM heap settings? > > 2) Do I need to decrease max heap size while using JNA? > > 3) How do I limit RAM allocated by JNA? > > 4) Where can I see / monitor row cache size? > > 5) I've configured JNA just for test on my dev computer and so far I've > noticed serious performance issues (high cpu usage on heavy write load), so > I must be doing something wrong I've just copied JNA jars into > Cassandra/lib, without installing any native libs. This should not work at > all, right? > > Thanks, > Maciej > >
RE: Cassandra 1.x and proper JNA setup
Up, also interested in answers to questions below. Best regards/ Pagarbiai Viktor Jevdokimov Senior Developer Email: viktor.jevdoki...@adform.com Phone: +370 5 212 3063 Fax: +370 5 261 0453 J. Jasinskio 16C, LT-01112 Vilnius, Lithuania Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.-Original Message- From: Maciej Miklas [mailto:mac.mik...@googlemail.com] Sent: Tuesday, November 01, 2011 11:15 To: user@cassandra.apache.org Subject: Cassandra 1.x and proper JNA setup Hi all, is there any documentation about proper JNA configuration? I do not understand few things: 1) Does JNA use JVM heap settings? 2) Do I need to decrease max heap size while using JNA? 3) How do I limit RAM allocated by JNA? 4) Where can I see / monitor row cache size? 5) I've configured JNA just for test on my dev computer and so far I've noticed serious performance issues (high cpu usage on heavy write load), so I must be doing something wrong I've just copied JNA jars into Cassandra/lib, without installing any native libs. This should not work at all, right? Thanks, Maciej
