Thank you for your help. We have been using jna-4.0.0.jar since using Cassandra 2.2(.6). Until last week, we had no issues. Now, we are experiencing the exception that I identified. We only have jna-4.0.0.jar loaded on our machines and the CLASSPATH that Cassandra builds only uses the jar from the /usr/share/cassandra/lib directory. Jna-3.2.4.jar was only introduced when I was debugging the original issue and will never be loaded on our machines. See the output of ps below.
Should I be looking elsewhere? Some have brought up that low memory on a JVM or LINUX machine could cause jar loading issues. I have tried doubling -Xss256k to -Xss512k. Thanks Billy S. Boutin Office Phone No. (913) 241-5574 Cell Phone No. (732) 213-1368 496 3318 1 0 Jul19 ? 00:06:20 /usr/java/latest/bin/java -Dorg.xerial.snappy.tempdir=/var/tmp -Djava.io.tmpdir=/var/tmp -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar -XX:+CMSClassUnloadingEnabled -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1968M -Xmx1968M -Xmn200M -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=1000003 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB -XX:+PerfDisableSharedMem -XX:CompileCommandFile=/etc/cassandra/conf/hotspot_compiler -XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:CMSWaitDuration=10000 -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -Xloggc:/usr/share/cassandra/logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Djava.net.preferIPv4Stack=true -Dcassandra.jmx.local.port=7199 -XX:+DisableExplicitGC -Djava.library.path=/usr/share/cassandra/lib/sigar-bin -Dlogback.configurationFile=logback.xml -Dcassandra.logdir=/usr/share/cassandra/logs -Dcassandra.storagedir= -Dcassandra-pidfile=/var/run/cassandra/cassandra.pid -cp /etc/cassandra/conf:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/apache-cassandra-clientutil-2.2.6-E002.jar:/usr/share/cassandra/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share /cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/crc32ex-0.1.1.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-16.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.23.Final.jar:/usr/share/cassandra/lib/ohc-core-0.3.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.3.4.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/super-c sv-2.1.0.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-2.2.6-E002.jar:/usr/share/cassandra/apache-cassandra-thrift-2.2.6-E002.jar:/usr/share/cassandra/stress.jar: org.apache.cassandra.service.CassandraDaemon -----Original Message----- From: Jeff Jirsa [mailto:jji...@apache.org] Sent: Wednesday, July 19, 2017 11:47 PM To: user@cassandra.apache.org Subject: Re: Cassandra 2.2.6 Fails to Boot Up correctly - JNA Class On 2017-07-19 10:41 (-0700), William Boutin <william.bou...@ericsson.com> wrote: > We are running apache-cassandra-2.2.6 for months with no JNA startup issues. > Recently, we have updated some of our cassandra machines and we ran into a > Cassandra startup issue with JNA. See the stack trace 1 below. > > Question 1: Our Cassandra package contains jna-4.0.0.jar in the > /usr/share/cassandra/lib/ directory. Does apache-cassandra-2.2.6 specifically > package A-C* with the jna-4.0.0.jar ? > Yes, we explicitly bundle jna-4.0.0.jar for 2.2.6 This is unchanged in 2.2 since https://github.com/apache/cassandra/commit/2ba394676dc673b6d66a07247dccd122b64b0578 ( Jan 2014, when it was cassandra 2.1 beta). Is the jar missing? Did another version of jna get installed? If a system package got installed, you may have multiple versions in the classpath. > I tried to circumvent the exception by yum installing jna-3.2.4.jar for RHEL > 6.4 and adding it to the /usr/share/cassandra/lib/ directory. This got me > past the JNA mlockall failure with "INFO 16:56:47 JNA mlockall successful" > but when the MemtableFlushWriter requested memory allocation, I now get the > STACK TRACE 2 error. "nodetool status" still reports the node as incomplete. > Installing 3.2.4 is probably not going to work (as hinted in my earlier paragraph); you almost certainly want 4.0 or newer, and you should probably want exactly one jna jar in the classpath at any given time. - Jeff --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
