Re: Cassandra - crash with “free() invalid pointer”

2012-04-02 Thread Vijay
Can you send us the stack trace which you can find in the hs_err_pid*.log?
is the system memory all used up (free)? any errors in the logs just before
the crash?

Regards,
/VJ



On Mon, Mar 26, 2012 at 12:35 AM, Maciej Miklas
mac.mik...@googlemail.comwrote:

 I have row cache - it's about 20GB big in this case.
 The problem can be reproduced with our load test - we are using 20 reader
 threads on single Cassandra node.

 I will retest it with Java 6 - still it looks to me like JNA problem and
 JDK in this case should not matter, but we will see.


 On Thu, Mar 22, 2012 at 8:27 PM, Benoit Perroud ben...@noisette.chwrote:

 Sounds like a race condition in the off heap caching while calling
 Unsafe.free().

 Do you use cache ? What is your use case when you encounter this error
 ? Are you able to reproduce it ?


 2012/3/22 Maciej Miklas mac.mik...@googlemail.com:
  Hi *,
 
  My Cassandra installation runs on flowing system:
 
  Linux with Kernel 2.6.32.22
  jna-3.3.0
  Java 1.7.0-b147
 
  Sometimes we are getting following error:
 
  *** glibc detected *** /var/opt/java1.7/bin/java: free(): invalid
 pointer:
  0x7f66088a6000 ***
  === Backtrace: =
  /lib/libc.so.6[0x7f661d7099a8]
  /lib/libc.so.6(cfree+0x76)[0x7f661d70bab6]
  /lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x59)[0x7f661e02f349]
  /lib/libpthread.so.0[0x7f661de09237]
  /lib/libpthread.so.0[0x7f661de0931a]
  /lib/libpthread.so.0[0x7f661de0a0bd]
  /lib/libc.so.6(clone+0x6d)[0x7f661d76564d]
  === Memory map: 
  0040-00401000 r-xp  68:07 537448203
  /var/opt/jdk1.7.0/bin/java
  0060-00601000 rw-p  68:07 537448203
  /var/opt/jdk1.7.0/bin/java
  01bae000-01fd rw-p  00:00 0
  [heap]
  01fd-15798000 rw-p  00:00 0
  [heap]
  40002000-40005000 ---p  00:00 0
  40005000-40023000 rw-p  00:00 0
  4003-40033000 ---p  00:00 0
  40033000-40051000 rw-p  00:00 0
 
  Does anyone have similar problems? or maybe some hints?
 
  Thanks,
  Maciej



 --
 sent from my Nokia 3210





Re: Cassandra - crash with “free() invalid pointer”

2012-03-22 Thread Benoit Perroud
Sounds like a race condition in the off heap caching while calling
Unsafe.free().

Do you use cache ? What is your use case when you encounter this error
? Are you able to reproduce it ?


2012/3/22 Maciej Miklas mac.mik...@googlemail.com:
 Hi *,

 My Cassandra installation runs on flowing system:

 Linux with Kernel 2.6.32.22
 jna-3.3.0
 Java 1.7.0-b147

 Sometimes we are getting following error:

 *** glibc detected *** /var/opt/java1.7/bin/java: free(): invalid pointer:
 0x7f66088a6000 ***
 === Backtrace: =
 /lib/libc.so.6[0x7f661d7099a8]
 /lib/libc.so.6(cfree+0x76)[0x7f661d70bab6]
 /lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x59)[0x7f661e02f349]
 /lib/libpthread.so.0[0x7f661de09237]
 /lib/libpthread.so.0[0x7f661de0931a]
 /lib/libpthread.so.0[0x7f661de0a0bd]
 /lib/libc.so.6(clone+0x6d)[0x7f661d76564d]
 === Memory map: 
 0040-00401000 r-xp  68:07 537448203
 /var/opt/jdk1.7.0/bin/java
 0060-00601000 rw-p  68:07 537448203
 /var/opt/jdk1.7.0/bin/java
 01bae000-01fd rw-p  00:00 0
 [heap]
 01fd-15798000 rw-p  00:00 0
 [heap]
 40002000-40005000 ---p  00:00 0
 40005000-40023000 rw-p  00:00 0
 4003-40033000 ---p  00:00 0
 40033000-40051000 rw-p  00:00 0

 Does anyone have similar problems? or maybe some hints?

 Thanks,
 Maciej



-- 
sent from my Nokia 3210