RE: [Cassandra 3.0.9] Cannot allocate memory
Thanks Jayesh, I found the fix for the same. I make below changes : In /etc/sysctl.conf I make below change: vm.max_map_count = 1048575 in the /etc/security/limits.d file: root - memlock unlimited root - nofile 10 root - nproc 32768 root - as unlimited Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: Thursday, March 23, 2017 8:36 PM To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in>; user@cassandra.apache.org Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Dmesg will often print a message saying that it had to kill a process if the server was short of memory, so you will have to dump the output to a file and check. If a process is killed to reclaim memory for the system, then it will dump a list of all processes and the actual process that was killed. So maybe, you can check for a kill like this - "dmesg | grep -i kill" If you do find a line (or two), then you need to examine the output carefully. In production, I tend to dump a lot of GC output also which helps troubleshooting. E.g. Below is what I have. If you look, I also have a flag that says that if the heap runs out of memory (which is rare), then dump files. If dmesg does not show your processes being killed, then you may have to dump gc logging info to get some insight. -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms16G -Xmx16G -Xmn4800M -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseParNewGC -XX:MaxTenuringThreshold=2 -XX:SurvivorRatio=8 -XX:+UnlockDiagnosticVMOptions -XX:ParGCCardsPerStrideChunk=32768 -XX:NewSize=750m -XX:MaxNewSize=750m -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -Xloggc: -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.ssl=<true|false> -Dcom.sun.management.jmxremote.authenticate=<true|false> From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 5:18 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory JVM config is as below: -Xms16G -Xmx16G -Xmn3000M What I need to check in dmesg? From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 23 March 2017 03:39 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory And what is the configured max heap? Sometimes you may also be able to see some useful messages in "dmesg" output. Jayesh From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Sent: Wednesday, March 22, 2017 5:05:14 PM To: Thakrar, Jayesh; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory No only Cassandra is running on these servers. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 22 March 2017 22:27 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory
Re: [Cassandra 3.0.9] Cannot allocate memory
Dmesg will often print a message saying that it had to kill a process if the server was short of memory, so you will have to dump the output to a file and check. If a process is killed to reclaim memory for the system, then it will dump a list of all processes and the actual process that was killed. So maybe, you can check for a kill like this - "dmesg | grep -i kill" If you do find a line (or two), then you need to examine the output carefully. In production, I tend to dump a lot of GC output also which helps troubleshooting. E.g. Below is what I have. If you look, I also have a flag that says that if the heap runs out of memory (which is rare), then dump files. If dmesg does not show your processes being killed, then you may have to dump gc logging info to get some insight. -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms16G -Xmx16G -Xmn4800M -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseParNewGC -XX:MaxTenuringThreshold=2 -XX:SurvivorRatio=8 -XX:+UnlockDiagnosticVMOptions -XX:ParGCCardsPerStrideChunk=32768 -XX:NewSize=750m -XX:MaxNewSize=750m -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -Xloggc: -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.ssl=<true|false> -Dcom.sun.management.jmxremote.authenticate=<true|false> From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in> Date: Wednesday, March 22, 2017 at 5:18 PM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory JVM config is as below: -Xms16G -Xmx16G -Xmn3000M What I need to check in dmesg? From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 23 March 2017 03:39 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in>; user@cassandra.apache.org Subject: RE: [Cassandra 3.0.9] Cannot allocate memory And what is the configured max heap? Sometimes you may also be able to see some useful messages in "dmesg" output. Jayesh From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Sent: Wednesday, March 22, 2017 5:05:14 PM To: Thakrar, Jayesh; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory No only Cassandra is running on these servers. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 22 March 2017 22:27 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Intern
Re: [Cassandra 3.0.9] Cannot allocate memory
On Thu, Mar 23, 2017 at 11:18 AM, Abhishek Kumar Maheshwari < abhishek.maheshw...@timesinternet.in> wrote: > JVM config is as below: > > > > -Xms16G > > -Xmx16G > > -Xmn3000M > > > I don't think it is the cause, but you need to remove Xmn when using G1GC.
RE: [Cassandra 3.0.9] Cannot allocate memory
JVM config is as below: -Xms16G -Xmx16G -Xmn3000M What I need to check in dmesg? From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 23 March 2017 03:39 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in>; user@cassandra.apache.org Subject: RE: [Cassandra 3.0.9] Cannot allocate memory And what is the configured max heap? Sometimes you may also be able to see some useful messages in "dmesg" output. Jayesh From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Sent: Wednesday, March 22, 2017 5:05:14 PM To: Thakrar, Jayesh; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: RE: [Cassandra 3.0.9] Cannot allocate memory No only Cassandra is running on these servers. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 22 March 2017 22:27 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>; user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com<http://www.et-gbs.com>
RE: [Cassandra 3.0.9] Cannot allocate memory
The exception is as below: INFO 17:42:37 Index build of til_lineitem_productsku_item_id,til_lineitem_productsku_status complete WARN 17:43:28 G1 Old Generation GC in 2906ms. G1 Eden Space: 1560281088 -> 0; G1 Old Gen: 3033393144 -> 1127339984; G1 Survivor Space : 150994944 -> 0; INFO 17:43:28 Pool NameActive Pending Completed Blocked All Time Blocked Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f9f31a55000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. # An error report file with more information is saved as: # /opt/apache-cassandra-3.0.9/bin/hs_err_pid8264.log INFO 17:43:28 MutationStage 1 0 55409726 0 0 INFO 17:43:28 ViewMutationStage 0 0 0 0 0 INFO 17:43:28 ReadStage 0 0 0 0 0 INFO 17:43:28 RequestResponseStage 0 0 17 0 0 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f9f830d8000, 65536, 1) failed; error='Cannot allocate memory' (errno=12)INFO 17:43:28 ReadRepairStage 0 0 0 0 0 [thread 140321932023552 also had an error] [thread 140321811961600 also had an error] ERROR 17:43:28 Exception in thread Thread[CompactionExecutor:2482,1,main] org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:156) ~[apache-cassandra-3.0.9.jar:3.0.9] at org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:280) ~[apache-cassandra-3.0.9.jar:3.0.9] at org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:216) ~[apache-cassandra-3.0.9.jar:3.0.9] From: Abhishek Kumar Maheshwari Sent: 22 March 2017 11:15 To: user@cassandra.apache.org Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness.
RE: [Cassandra 3.0.9] Cannot allocate memory
And what is the configured max heap? Sometimes you may also be able to see some useful messages in "dmesg" output. Jayesh From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in> Sent: Wednesday, March 22, 2017 5:05:14 PM To: Thakrar, Jayesh; user@cassandra.apache.org Subject: RE: [Cassandra 3.0.9] Cannot allocate memory No only Cassandra is running on these servers. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 22 March 2017 22:27 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in>; user@cassandra.apache.org Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com<http://www.et-gbs.com>
RE: [Cassandra 3.0.9] Cannot allocate memory
No only Cassandra is running on these servers. From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: 22 March 2017 22:27 To: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in>; user@cassandra.apache.org Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com<http://www.et-gbs.com>
RE: [Cassandra 3.0.9] Cannot allocate memory
Hi Abhishek, In sysctl.conf file we have below setting: vm.swappiness = 10 vm.dirty_ratio = 60 vm.dirty_background_ratio = 2 vm.max_map_count = 1048575 so I need to apply patch for same? From: Abhishek Verma [mailto:ve...@uber.com] Sent: 22 March 2017 23:04 To: user@cassandra.apache.org Cc: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in> Subject: Re: [Cassandra 3.0.9] Cannot allocate memory Just a shot in the dark, but what is your setting of vm.max_map_count in /etc/sysctl.conf ? It is recommended to set it to: vm.max_map_count = 1048575 Source: https://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html We saw a similar problem in the past where mmap failed and we added a check to emit warning as a part of https://issues.apache.org/jira/browse/CASSANDRA-13008. On Wed, Mar 22, 2017 at 9:57 AM, Thakrar, Jayesh <jthak...@conversantmedia.com<mailto:jthak...@conversantmedia.com>> wrote: Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591<tel:+91%208%2005591> (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com<http://www.et-gbs.com>
Re: [Cassandra 3.0.9] Cannot allocate memory
Just a shot in the dark, but what is your setting of vm.max_map_count in /etc/sysctl.conf ? It is recommended to set it to: vm.max_map_count = 1048575 Source: https://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html We saw a similar problem in the past where mmap failed and we added a check to emit warning as a part of https://issues.apache.org/jira/browse/CASSANDRA-13008. On Wed, Mar 22, 2017 at 9:57 AM, Thakrar, Jayesh < jthak...@conversantmedia.com> wrote: > Is/are the Cassandra server(s) shared? > > E.g. do they run mesos + spark? > > > > *From: *Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in> > *Date: *Wednesday, March 22, 2017 at 12:45 AM > *To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Subject: *[Cassandra 3.0.9] Cannot allocate memory > > > > Hi all, > > > > I am using Cassandra 3.0.9. while I am adding new server after some time I > am getting below exception. JVM option is attaches. > > Hardware info: > > Ram 64 GB. > > Core: 40 > > > > > > Java HotSpot(TM) 64-Bit Server VM warning: INFO: > os::commit_memory(0x7fe9c44ee000, > 12288, 0) failed; error='Cannot allocate memory' (errno=12) > > # > > # There is insufficient memory for the Java Runtime Environment to > continue. > > # Native memory allocation (mmap) failed to map 12288 bytes for committing > reserved memory. > > Java HotSpot(TM) 64-Bit Server VM warning: INFO: > os::commit_memory(0x7f5c056ab000, > 12288, 0) failed; error='Cannot allocate memory' (errno=12) > > [thread 140033204860672 also had an error] > > Java HotSpot(TM) 64-Bit Server VM warning: INFO: > os::commit_memory(0x7f5c0566a000, > 12288, 0) failed; error='Cannot allocate memory' (errno=12) > > [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server > VM warning: > > INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; > error='Cannot allocate memory' (errno=12) > > Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also > had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) > failed; error='Cannot allocate memory' (errno=12) > > > > Please let me know what I miss? > > > > *Thanks & Regards,* > *Abhishek Kumar Maheshwari* > *+91- 805591 <+91%208%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > YES Bank & The Economic Times Global Business Summit (GBS) is back on > 27-28 March. The 3rd edition of GBS will see participation of 2000+ > delegates from over 20 countries as they bear witness to leaders sharing > insights into how to best navigate a dynamic future. Visit www.et-gbs.com >
Re: [Cassandra 3.0.9] Cannot allocate memory
Is/are the Cassandra server(s) shared? E.g. do they run mesos + spark? From: Abhishek Kumar Maheshwari <abhishek.maheshw...@timesinternet.in> Date: Wednesday, March 22, 2017 at 12:45 AM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: [Cassandra 3.0.9] Cannot allocate memory Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com
[Cassandra 3.0.9] Cannot allocate memory
Hi all, I am using Cassandra 3.0.9. while I am adding new server after some time I am getting below exception. JVM option is attaches. Hardware info: Ram 64 GB. Core: 40 Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c44ee000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory. Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c056ab000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204860672 also had an error] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7f5c0566a000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) [thread 140033204594432 also had an error]Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x7fe9c420c000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Java HotSpot(TM) 64-Bit Server VM warning: [thread 140641994852096 also had an error]INFO: os::commit_memory(0x7f5c055a7000, 12288, 0) failed; error='Cannot allocate memory' (errno=12) Please let me know what I miss? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. YES Bank & The Economic Times Global Business Summit (GBS) is back on 27-28 March. The 3rd edition of GBS will see participation of 2000+ delegates from over 20 countries as they bear witness to leaders sharing insights into how to best navigate a dynamic future. Visit www.et-gbs.com jvm.options Description: jvm.options
