Hi Pete!
Thank you for your response. "Have you tried using the supplied hdfs_write executable includes in the distribution?"I wrote a similar one to perform my tests. I'm gonna try this default one. I'll get back to you with the results. "I didn't understand your comment about using hdfsJniHelper.c - that should be used only by hdfs.c itself." That is correct. But the libhdfs shared library contains hdfs.c implementation so when calling libhdfs functions than the JNI layer in hdfsJniHelper.c will fork a new JVM and performs the appropriate Java calls to hdfs. Please correct me if I'm wrong. Since the SIGSEGV error comes in case of rlimit stack problem I thought that I should supply additional memory parameters to the JVM: -Xss256M -Xoss256M -XX:ThreadStackSize=262144 -Xms128M -Xmx128M I added them as extra parameters to JavaVMOption which is used by JNI_CreateJavaVM function. "Also, what version of hadoop is this?" I'm using hadoop-0.19 from the svn. Today I will perform tests on the latest stable 0.18.2 version. If you have seen this SIGSEGV error formerly could you describe me the former reason and the solution/workaround? Thank you. Cheers, Tamas --- On Thu, 11/6/08, Pete Wyckoff <[EMAIL PROTECTED]> wrote: From: Pete Wyckoff <[EMAIL PROTECTED]> Subject: Re: libhdfs SIGSEGV error To: "," <[EMAIL PROTECTED]@yahoo.com>, "[email protected]" <[email protected]> Date: Thursday, November 6, 2008, 7:20 PM Hi Tamas, Have you tried using the supplied hdfs_write executable includes in the distribution? Also, I didn't understand your comment about using hdfsJniHelper.c - that should be used only by hdfs.c itself. Also, what version of hadoop is this? I haven't seen this problem at least in hadoop 0.17. And have you run this under gdb? -- pete On 11/6/08 10:30 AM, "Tamás Szokol" <[EMAIL PROTECTED]> wrote: Hello! I'd like to ask your help in a libhdfs related problem. I'm trying to perform HDFS tests from C by using the libhdfs API. I created a test program, that measures the creation times of 1MB, 10MB, 100MB and 1GB large files. The test runs well for 1 MB and 10 MB but as soon as I reach to 100MB I receive a SIGSEGV error: ================================================================== # # An unexpected error has been detected by Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007fbead12a32c, pid=6918, tid=140456938362592 # # Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b22 mixed mode linux-amd64) # Problematic frame: # V [libjvm.so+0x36d32c] # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x0000000000609800): JavaThread "main" [_thread_in_vm, id=6918, stack(0x00007fffb5cc2000,0x00007fffb5ec2000)] siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000000 Registers: RAX=0x0000000000000090, RBX=0x0000000000609800, RCX=0x00007fbead60d1e0, RDX=0x00007fbead447780 RSP=0x00007fffb5ec1240, RBP=0x00007fffb5ec12c0, RSI=0x0000000000000000, RDI=0x0000000000609800 R8 =0x00007fbe7dcff010, R9 =0x0000000000000400, R10=0x00007fbead610c50, R11=0x00007fbea35180c0 R12=0x0000000006300000, R13=0x0000000000000000, R14=0x0000000000000000, R15=0x00007fffb5ec1250 RIP=0x00007fbead12a32c, EFL=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004 TRAPNO=0x000000000000000e Top of Stack: (sp=0x00007fffb5ec1240) 0x00007fffb5ec1240: 0000000000609998 00007fbe7dcff010 0x00007fffb5ec1250: 0000000000609800 000000000060a1f0 0x00007fffb5ec1260: 0000000000000090 00007fbead447780 0x00007fffb5ec1270: 0000000000609800 0000000000602720 0x00007fffb5ec1280: 0000000000609800 0000000000609998 0x00007fffb5ec1290: 0000000000609800 00007fbe84014220 0x00007fffb5ec12a0: 0000000000609998 00007fbe7dcff010 0x00007fffb5ec12b0: 00000000006f7848 0000000000000000 0x00007fffb5ec12c0: 0000000006300000 00007fbeada9ccff 0x00007fffb5ec12d0: 00000000006f7830 0000000000000000 0x00007fffb5ec12e0: 000000000060a660 0000000000000000 0x00007fffb5ec12f0: 0000000000000000 0000000000000021 0x00007fffb5ec1300: 00007fffb5ec1370 0000000000000000 0x00007fffb5ec1310: 00007fffb5ec14b0 0000000000000000 0x00007fffb5ec1320: 0000000000000000 0000000000400e37 0x00007fffb5ec1330: 0000000000000000 00007fbe7dcff010 0x00007fffb5ec1340: 0630000000000001 0000000000602720 0x00007fffb5ec1350: 063000000000000b 00007fbe84014220 0x00007fffb5ec1360: 00000000006f7840 0000000000000021 0x00007fffb5ec1370: 00007fffb5ec1390 0000000000400cdd 0x00007fffb5ec1380: 00007fbe7dcff010 0000001800401126 0x00007fffb5ec1390: 00007fffb5ec13d0 0000000000400b1b 0x00007fffb5ec13a0: 000000000000003c 00007fffb5ec14b8 0x00007fffb5ec13b0: 0000000100401040 0000000000602280 0x00007fffb5ec13c0: 0000000000000022 00007fbeadec0c00 0x00007fffb5ec13d0: 0000000000000000 00007fbead7561c4 0x00007fffb5ec13e0: 00000000004009c0 00007fffb5ec14b8 0x00007fffb5ec13f0: 0000000100000000 0000000000400a78 0x00007fffb5ec1400: 00007fbeadec0c00 0c1103e05d01fe14 0x00007fffb5ec1410: 0000000000000000 00007fffb5ec14b0 0x00007fffb5ec1420: 0000000000000000 0000000000000000 0x00007fffb5ec1430: f3ee68387ac1fe14 f36c590a9ffffe14 Instructions: (pc=0x00007fbead12a32c) 0x00007fbead12a31c: 5d 90 48 83 7b 08 00 0f 85 5d 01 00 00 45 85 ed 0x00007fbead12a32c: 49 8b 36 78 13 45 85 e4 78 0e 47 8d 1c 2c 44 3b Stack: [0x00007fffb5cc2000,0x00007fffb5ec2000], sp=0x00007fffb5ec1240, free space=2044k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x36d32c] --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x000000000069f800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=6925, stack(0x00007fbe8b1b5000,0x00007fbe9b1b6000)] 0x000000000069dc00 JavaThread "CompilerThread1" daemon [_thread_blocked, id=6924, stack(0x0000000041c3e000,0x0000000041d3f000)] 0x0000000000692400 JavaThread "CompilerThread0" daemon [_thread_blocked, id=6923, stack(0x0000000041b3d000,0x0000000041c3e000)] 0x0000000000690c00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6922, stack(0x0000000061e4c000,0x0000000071e4d000)] 0x000000000066cc00 JavaThread "Finalizer" daemon [_thread_blocked, id=6921, stack(0x0000000051e4b000,0x0000000061e4c000)] 0x000000000066b800 JavaThread "Reference Handler" daemon [_thread_blocked, id=6920, stack(0x0000000041e4a000,0x0000000051e4b000)] =>0x0000000000609800 JavaThread "main" [_thread_in_vm, id=6918, stack(0x00007fffb5cc2000,0x00007fffb5ec2000)] Other Threads: 0x0000000000666400 VMThread [stack: 0x0000000041a3c000,0x0000000041b3d000] [id=6919] 0x00000000006a1800 WatcherThread [stack: 0x0000000041d3f000,0x0000000041e40000] [id=6926] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap def new generation total 39296K, used 0K [0x00007fbe9b470000, 0x00007fbe9df10000, 0x00007fbe9df10000) eden space 34944K, 0% used [0x00007fbe9b470000, 0x00007fbe9b470000, 0x00007fbe9d690000) from space 4352K, 0% used [0x00007fbe9d690000, 0x00007fbe9d690000, 0x00007fbe9dad0000) to space 4352K, 0% used [0x00007fbe9dad0000, 0x00007fbe9dad0000, 0x00007fbe9df10000) tenured generation total 87424K, used 405K [0x00007fbe9df10000, 0x00007fbea3470000, 0x00007fbea3470000) the space 87424K, 0% used [0x00007fbe9df10000, 0x00007fbe9df75778, 0x00007fbe9df75800, 0x00007fbea3470000) compacting perm gen total 21248K, used 6148K [0x00007fbea3470000, 0x00007fbea4930000, 0x00007fbea8870000) the space 21248K, 28% used [0x00007fbea3470000, 0x00007fbea3a71280, 0x00007fbea3a71400, 0x00007fbea4930000) No shared spaces configured. Dynamic libraries: 00400000-00402000 r-xp 00000000 08:01 107801 /home/sony/dev/hadoop-0.19/libhdfs/hdfs_write_test2 00601000-00602000 rw-p 00001000 08:01 107801 /home/sony/dev/hadoop-0.19/libhdfs/hdfs_write_test2 00602000-01b8f000 rw-p 00602000 00:00 0 [heap] 41a3c000-41a3d000 ---p 41a3c000 00:00 0 41a3d000-41b3d000 rwxp 41a3d000 00:00 0 41b3d000-41b40000 ---p 41b3d000 00:00 0 41b40000-41c3e000 rwxp 41b40000 00:00 0 41c3e000-41c41000 ---p 41c3e000 00:00 0 41c41000-41d3f000 rwxp 41c41000 00:00 0 41d3f000-41d40000 ---p 41d3f000 00:00 0 41d40000-41e40000 rwxp 41d40000 00:00 0 41e4a000-41e4d000 ---p 41e4a000 00:00 0 41e4d000-51e4b000 rwxp 41e4d000 00:00 0 51e4b000-51e4e000 ---p 51e4b000 00:00 0 51e4e000-61e4c000 rwxp 51e4e000 00:00 0 61e4c000-61e4f000 ---p 61e4c000 00:00 0 61e4f000-71e4d000 rwxp 61e4f000 00:00 0 7fbe50dfc000-7fbe50dff000 rwxp 7fbe50dfc000 00:00 0 7fbe50dff000-7fbe60dfd000 rwxp 7fbe50dff000 00:00 0 7fbe7dcff000-7fbe840cd000 rw-p 7fbe7dcff000 00:00 0 7fbe840cd000-7fbe88000000 ---p 7fbe840cd000 00:00 0 7fbe8b08d000-7fbe8b093000 r-xp 00000000 08:01 466691 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libmanagement.so 7fbe8b093000-7fbe8b192000 ---p 00006000 08:01 466691 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libmanagement.so 7fbe8b192000-7fbe8b194000 rw-p 00005000 08:01 466691 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libmanagement.so 7fbe8b194000-7fbe8b1b5000 r--s 0022f000 08:01 101826 /home/sony/dev/hadoop-0.19/build/hadoop-0.20.0-dev-core.jar 7fbe8b1b5000-7fbe8b1b8000 ---p 7fbe8b1b5000 00:00 0 7fbe8b1b8000-7fbe9b1b6000 rwxp 7fbe8b1b8000 00:00 0 7fbe9b1b6000-7fbe9b1f5000 r--p 00000000 08:01 466319 /usr/lib/locale/hu_HU.utf8/LC_CTYPE 7fbe9b1f5000-7fbe9b227000 rw-p 7fbe9b1f5000 00:00 0 7fbe9b227000-7fbe9b3b2000 r--s 02df0000 08:01 434280 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/rt.jar 7fbe9b3b2000-7fbe9b3da000 rw-p 7fbe9b3b2000 00:00 0 7fbe9b3da000-7fbe9b3e5000 rwxp 7fbe9b3da000 00:00 0 7fbe9b3e5000-7fbe9b405000 rwxp 7fbe9b3e5000 00:00 0 7fbe9b405000-7fbe9b450000 rwxp 7fbe9b405000 00:00 0 7fbe9b450000-7fbe9b46f000 rwxp 7fbe9b450000 00:00 0 7fbe9b46f000-7fbea4930000 rwxp 7fbe9b46f000 00:00 0 7fbea4930000-7fbea8870000 rwxp 7fbea4930000 00:00 0 7fbea8874000-7fbea8876000 r--s 0000d000 08:01 181160 /home/sony/dev/commons-logging-1.1.1/commons-logging-1.1.1.jar 7fbea8876000-7fbea887d000 r--s 00000000 08:01 447736 /usr/lib/gconv/gconv-modules.cache 7fbea887d000-7fbea8aed000 rwxp 7fbea887d000 00:00 0 7fbea8aed000-7fbeab87d000 rwxp 7fbea8aed000 00:00 0 7fbeab87d000-7fbeab88b000 r-xp 00000000 08:01 466659 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libzip.so 7fbeab88b000-7fbeab98d000 ---p 0000e000 08:01 466659 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libzip.so 7fbeab98d000-7fbeab990000 rw-p 00010000 08:01 466659 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libzip.so 7fbeab990000-7fbeab991000 rw-p 7fbeab990000 00:00 0 7fbeab991000-7fbeab9b9000 r-xp 00000000 08:01 466672 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libjava.so 7fbeab9b9000-7fbeabab9000 ---p 00028000 08:01 466672 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libjava.so 7fbeabab9000-7fbeabac0000 rw-p 00028000 08:01 466672 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libjava.so 7fbeabac0000-7fbeabacd000 r-xp 00000000 08:01 466654 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libverify.so 7fbeabacd000-7fbeabbcc000 ---p 0000d000 08:01 466654 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libverify.so 7fbeabbcc000-7fbeabbcf000 rw-p 0000c000 08:01 466654 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/libverify.so 7fbeabbcf000-7fbeabbd9000 r-xp 00000000 08:01 574895 /lib/libnss_files-2.7.so 7fbeabbd9000-7fbeabdd9000 ---p 0000a000 08:01 574895 /lib/libnss_files-2.7.so 7fbeabdd9000-7fbeabddb000 rw-p 0000a000 08:01 574895 /lib/libnss_files-2.7.so 7fbeabddb000-7fbeabde5000 r-xp 00000000 08:01 574897 /lib/libnss_nis-2.7.so 7fbeabde5000-7fbeabfe4000 ---p 0000a000 08:01 574897 /lib/libnss_nis-2.7.so 7fbeabfe4000-7fbeabfe6000 rw-p 00009000 08:01 574897 /lib/libnss_nis-2.7.so 7fbeabfe6000-7fbeabfee000 r-xp 00000000 08:01 574893 /lib/libnss_compat-2.7.so 7fbeabfee000-7fbeac1ed000 ---p 00008000 08:01 574893 /lib/libnss_compat-2.7.so 7fbeac1ed000-7fbeac1ef000 rw-p 00007000 08:01 574893 /lib/libnss_compat-2.7.so 7fbeac1ef000-7fbeac205000 r-xp 00000000 08:01 574892 /lib/libnsl-2.7.so 7fbeac205000-7fbeac404000 ---p 00016000 08:01 574892 /lib/libnsl-2.7.so 7fbeac404000-7fbeac406000 rw-p 00015000 08:01 574892 /lib/libnsl-2.7.so 7fbeac406000-7fbeac408000 rw-p 7fbeac406000 00:00 0 7fbeac408000-7fbeac40f000 r-xp 00000000 08:01 466661 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/native_threads/libhpi.so 7fbeac40f000-7fbeac510000 ---p 00007000 08:01 466661 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/native_threads/libhpi.so 7fbeac510000-7fbeac512000 rw-p 00008000 08:01 466661 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/native_threads/libhpi.so 7fbeac512000-7fbeac513000 rw-p 7fbeac512000 00:00 0 7fbeac513000-7fbeac51b000 r-xp 00000000 08:01 574902 /lib/librt-2.7.so 7fbeac51b000-7fbeac71a000 ---p 00008000 08:01 574902 /lib/librt-2.7.so 7fbeac71a000-7fbeac71c000 rw-p 00007000 08:01 574902 /lib/librt-2.7.so 7fbeac71c000-7fbeac732000 r-xp 00000000 08:01 574900 /lib/libpthread-2.7.so 7fbeac732000-7fbeac932000 ---p 00016000 08:01 574900 /lib/libpthread-2.7.so 7fbeac932000-7fbeac934000 rw-p 00016000 08:01 574900 /lib/libpthread-2.7.so 7fbeac934000-7fbeac938000 rw-p 7fbeac934000 00:00 0 7fbeac938000-7fbeac93a000 r-xp 00000000 08:01 574889 /lib/libdl-2.7.so 7fbeac93a000-7fbeacb3a000 ---p 00002000 08:01 574889 /lib/libdl-2.7.so 7fbeacb3a000-7fbeacb3c000 rw-p 00002000 08:01 574889 /lib/libdl-2.7.so 7fbeacb3c000-7fbeacbbc000 r-xp 00000000 08:01 574890 /lib/libm-2.7.so 7fbeacbbc000-7fbeacdbb000 ---p 00080000 08:01 574890 /lib/libm-2.7.so 7fbeacdbb000-7fbeacdbd000 rw-p 0007f000 08:01 574890 /lib/libm-2.7.so 7fbeacdbd000-7fbead4b2000 r-xp 00000000 08:01 466678 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/server/libjvm.so 7fbead4b2000-7fbead5b1000 ---p 006f5000 08:01 466678 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/server/libjvm.so 7fbead5b1000-7fbead6fb000 rw-p 006f4000 08:01 466678 /usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/server/libjvm.so 7fbead6fb000-7fbead738000 rw-p 7fbead6fb000 00:00 0 7fbead738000-7fbead890000 r-xp 00000000 08:01 574886 /lib/libc-2.7.so 7fbead890000-7fbeada90000 ---p 00158000 08:01 574886 /lib/libc-2.7.so 7fbeada90000-7fbeada93000 r--p 00158000 08:01 574886 /lib/libc-2.7.so 7fbeada93000-7fbeada95000 rw-p 0015b000 08:01 574886 /lib/libc-2.7.so 7fbeada95000-7fbeada9a000 rw-p 7fbeada95000 00:00 0 7fbeada9a000-7fbeadaa3000 r-xp 00000000 08:01 106434 /home/sony/dev/hadoop-0.19/libhdfs/libhdfs.so. 7fbeadaa3000-7fbeadca2000 ---p 00009000 08:01 106434 /home/sony/dev/hadoop-0.19/libhdfs/libhdfs.so. 7fbeadca2000-7fbeadca3000 rw-p 00008000 08:01 106434 /home/sony/dev/hadoop-0.19/libhdfs/libhdfs.so. 7fbeadca3000-7fbeadcc0000 r-xp 00000000 08:01 574883 /lib/ld-2.7.so 7fbeadcc1000-7fbeadcf6000 rwxp 7fbeadcc1000 00:00 0 7fbeadcf6000-7fbeaddac000 rwxp 7fbeadcf6000 00:00 0 7fbeaddac000-7fbeadeb1000 rw-p 7fbeaddac000 00:00 0 7fbeadeb1000-7fbeadeb2000 r--p 7fbeadeb1000 00:00 0 7fbeadeb2000-7fbeadeba000 rw-s 00000000 08:01 163163 /tmp/hsperfdata_sony/6918 7fbeadeba000-7fbeadebb000 rwxp 7fbeadeba000 00:00 0 7fbeadebb000-7fbeadebc000 r--p 7fbeadebb000 00:00 0 7fbeadebc000-7fbeadec0000 rw-p 7fbeadebc000 00:00 0 7fbeadec0000-7fbeadec2000 rw-p 0001d000 08:01 574883 /lib/ld-2.7.so 7fffb5cc2000-7fffb5cc5000 ---p 7fffb5cc2000 00:00 0 7fffb5cc5000-7fffb5ec2000 rwxp 7fffffe02000 00:00 0 [stack] 7fffb5ffe000-7fffb6000000 r-xp 7fffb5ffe000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] VM Arguments: jvm_args: -Xss256M -Xoss256M -XX:ThreadStackSize=262144 -Xms128M -Xmx128M java_command: <unknown> Launcher Type: generic Environment Variables: JAVA_HOME=/usr/lib/jvm/java-6-sun CLASSPATH=/usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/rt.jar:/home/sony/dev/hadoop-0.19/build/hadoop-0.20.0-dev-core.jar:/home/sony/dev/commons-logging-1.1.1/commons-logging-1.1.1.jar PATH=/home/sony/dev/hadoop-0.19/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games LD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.06/jre/lib/amd64/server:/home/sony/dev/hadoop-0.19/libhdfs SHELL=/bin/bash Signal Handlers: SIGSEGV: [libjvm.so+0x6616e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGBUS: [libjvm.so+0x6616e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGFPE: [libjvm.so+0x5044d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGPIPE: [libjvm.so+0x5044d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGXFSZ: [libjvm.so+0x5044d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGILL: [libjvm.so+0x5044d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000 SIGUSR2: [libjvm.so+0x506740], sa_mask[0]=0x00000000, sa_flags=0x10000004 SIGHUP: [libjvm.so+0x5064a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGINT: [libjvm.so+0x5064a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGTERM: [libjvm.so+0x5064a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 SIGQUIT: [libjvm.so+0x5064a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004 --------------- S Y S T E M --------------- OS:lenny/sid uname:Linux 2.6.24-19-generic #1 SMP Wed Aug 20 17:53:40 UTC 2008 x86_64 libc:glibc 2.7 NPTL 2.7 rlimit: STACK 1048576k, CORE 0k, NPROC 20480, NOFILE 1024, AS infinity load average:1.23 0.44 0.20 CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 15 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3 Memory: 4k page, physical 2580404k(957956k free), swap 489940k(489940k free) vm_info: Java HotSpot(TM) 64-Bit Server VM (10.0-b22) for linux-amd64 JRE (1.6.0_06-b02), built on Mar 25 2008 01:03:02 by "java_re" with gcc 3.2.2 (SuSE Linux) time: Thu Nov 6 11:19:55 2008 elapsed time: 29 seconds ================================================================== My test program uses the JniHelper.h and .c files from hadoop-0.19/src/c++/libhdfs. The test program creates buffers to be written out by hdfsWrite. I thought that the main thread's stack is not enough to handle this big buffers so I set the following parameters for the forked JVM: jvm_args: -Xss256M -Xoss256M -XX:ThreadStackSize=262144 -Xms128M -Xmx128M >From the log it can be seen that the main thread's stack still remains 2MB, though it should be 256MB. I played a bit with the file sizes and as it turned out it works for 1MB, 10 MB, 50 MB, 75MB, doesn't work for 99MB, 100MB, and above. Environmental information. I use hadoop-0.19 from the SVN repository. Ubuntu Gutsy Linux: Linux 2.6.24-19-generic #1 SMP Wed Aug 20 17:53:40 UTC 2008 x86_64 GNU/Linux I run linux under VMware. I use Java 6. I saw previously on the list libhdfs problems with SIGSEGV but I couldn't find the answer to solve my problem.. Thank you in advance for helping me. Cheers, Tamas
