[ https://issues.apache.org/jira/browse/SPARK-15843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-15843. ------------------------------- Resolution: Invalid Target Version/s: (was: 1.6.1) Please read https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark and ask questions at user@. This means your JVM ran out of OS memory, probably because you asked for more heap than the OS can allocate and perhaps swap is off. It's not a Spark issue. > Spark RAM issue > --------------- > > Key: SPARK-15843 > URL: https://issues.apache.org/jira/browse/SPARK-15843 > Project: Spark > Issue Type: Question > Components: Spark Shell > Affects Versions: 1.6.1 > Environment: RASPBIAN JESSIE > Full desktop image based on Debian Jessie > Reporter: Sreetej Lakkam > Labels: beginner, newbie > > Trying to run Spark 1.6.1 on Hadoop 2.7.1 over Raspberry Pi - 3. On > submitting spark-shell > sudo /opt/spark-1.6.1-bin-hadoop2.6/bin/spark-shell > produces an error > Java HotSpot(TM) Client VM warning: INFO: os::commit_memory(0x49540000, > 715915264, 0) failed; error='Cannot allocate memory' (errno=12) > # > # There is insufficient memory for the Java Runtime Environment to continue. > # Native memory allocation (mmap) failed to map 715915264 bytes for > committing reserved memory. > # An error report file with more information is saved as: > # /home/pi/hs_err_pid2179.log > Find the log file hs_err_pid2179.log > # > # There is insufficient memory for the Java Runtime Environment to continue. > # Native memory allocation (mmap) failed to map 715915264 bytes for > committing reserved memory. > # Possible reasons: > # The system is out of physical RAM or swap space > # In 32 bit mode, the process size limit was hit > # Possible solutions: > # Reduce memory load on the system > # Increase physical memory or swap space > # Check if swap backing store is full > # Use 64 bit Java on a 64 bit OS > # Decrease Java heap size (-Xmx/-Xms) > # Decrease number of Java threads > # Decrease Java thread stack sizes (-Xss) > # Set larger code cache with -XX:ReservedCodeCacheSize= > # This output file may be truncated or incomplete. > # > # Out of Memory Error (os_linux.cpp:2627), pid=2179, tid=1983399008 > # > # JRE version: (8.0_65-b17) (build ) > # Java VM: Java HotSpot(TM) Client VM (25.65-b01 mixed mode, sharing > linux-arm ) > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > --------------- T H R E A D --------------- > Current thread (0x76207400): JavaThread "Unknown thread" [_thread_in_vm, > id=2200, stack(0x76335000,0x76385000)] > Stack: [0x76335000,0x76385000] > --------------- P R O C E S S --------------- > Java Threads: ( => current thread ) > Other Threads: > =>0x76207400 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=2200, > stack(0x76335000,0x76385000)] > VM state:not at safepoint (not fully initialized) > VM Mutex/Monitor currently owned by a thread: None > GC Heap History (0 events): > No events > Deoptimization events (0 events): > No events > Internal exceptions (0 events): > No events > Events (0 events): > No events > Dynamic libraries: > 00008000-00009000 r-xp 00000000 b3:02 22725 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/bin/java > 00010000-00011000 rw-p 00000000 b3:02 22725 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/bin/java > 003e5000-00406000 rw-p 00000000 00:00 0 [heap] > 33dff000-33eaa000 rw-p 00000000 00:00 0 > 33eaa000-33fff000 ---p 00000000 00:00 0 > 33fff000-49540000 rw-p 00000000 00:00 0 > 740c3000-740c4000 rw-p 00000000 00:00 0 > 740c4000-74143000 ---p 00000000 00:00 0 > 74143000-7416b000 rwxp 00000000 00:00 0 > 7416b000-76143000 ---p 00000000 00:00 0 > 76143000-7615a000 r-xp 00000000 b3:02 22812 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libzip.so > 7615a000-76161000 ---p 00017000 b3:02 22812 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libzip.so > 76161000-76162000 rw-p 00016000 b3:02 22812 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libzip.so > 76162000-7616d000 r-xp 00000000 b3:02 3196 > /lib/arm-linux-gnueabihf/libnss_files-2.19.so > 7616d000-7617c000 ---p 0000b000 b3:02 3196 > /lib/arm-linux-gnueabihf/libnss_files-2.19.so > 7617c000-7617d000 r--p 0000a000 b3:02 3196 > /lib/arm-linux-gnueabihf/libnss_files-2.19.so > 7617d000-7617e000 rw-p 0000b000 b3:02 3196 > /lib/arm-linux-gnueabihf/libnss_files-2.19.so > 7617e000-76187000 r-xp 00000000 b3:02 3204 > /lib/arm-linux-gnueabihf/libnss_nis-2.19.so > 76187000-76196000 ---p 00009000 b3:02 3204 > /lib/arm-linux-gnueabihf/libnss_nis-2.19.so > 76196000-76197000 r--p 00008000 b3:02 3204 > /lib/arm-linux-gnueabihf/libnss_nis-2.19.so > 76197000-76198000 rw-p 00009000 b3:02 3204 > /lib/arm-linux-gnueabihf/libnss_nis-2.19.so > 76198000-761a9000 r-xp 00000000 b3:02 3193 > /lib/arm-linux-gnueabihf/libnsl-2.19.so > 761a9000-761b8000 ---p 00011000 b3:02 3193 > /lib/arm-linux-gnueabihf/libnsl-2.19.so > 761b8000-761b9000 r--p 00010000 b3:02 3193 > /lib/arm-linux-gnueabihf/libnsl-2.19.so > 761b9000-761ba000 rw-p 00011000 b3:02 3193 > /lib/arm-linux-gnueabihf/libnsl-2.19.so > 761ba000-761bc000 rw-p 00000000 00:00 0 > 761bc000-761c3000 r-xp 00000000 b3:02 3194 > /lib/arm-linux-gnueabihf/libnss_compat-2.19.so > 761c3000-761d2000 ---p 00007000 b3:02 3194 > /lib/arm-linux-gnueabihf/libnss_compat-2.19.so > 761d2000-761d3000 r--p 00006000 b3:02 3194 > /lib/arm-linux-gnueabihf/libnss_compat-2.19.so > 761d3000-761d4000 rw-p 00007000 b3:02 3194 > /lib/arm-linux-gnueabihf/libnss_compat-2.19.so > 761d4000-761f7000 r-xp 00000000 b3:02 22787 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libjava.so > 761f7000-761ff000 ---p 00023000 b3:02 22787 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libjava.so > 761ff000-76200000 rw-p 00023000 b3:02 22787 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libjava.so > 76200000-76221000 rw-p 00000000 00:00 0 > 76221000-76300000 ---p 00000000 00:00 0 > 7631e000-76324000 r-xp 00000000 b3:02 3223 > /lib/arm-linux-gnueabihf/librt-2.19.so > 76324000-76333000 ---p 00006000 b3:02 3223 > /lib/arm-linux-gnueabihf/librt-2.19.so > 76333000-76334000 r--p 00005000 b3:02 3223 > /lib/arm-linux-gnueabihf/librt-2.19.so > 76334000-76335000 rw-p 00006000 b3:02 3223 > /lib/arm-linux-gnueabihf/librt-2.19.so > 76335000-76338000 ---p 00000000 00:00 0 > 76338000-76385000 rwxp 00000000 00:00 0 [stack:2200] > 76385000-763ee000 r-xp 00000000 b3:02 3185 > /lib/arm-linux-gnueabihf/libm-2.19.so > 763ee000-763fe000 ---p 00069000 b3:02 3185 > /lib/arm-linux-gnueabihf/libm-2.19.so > 763fe000-763ff000 r--p 00069000 b3:02 3185 > /lib/arm-linux-gnueabihf/libm-2.19.so > 763ff000-76400000 rw-p 0006a000 b3:02 3185 > /lib/arm-linux-gnueabihf/libm-2.19.so > 76400000-76983000 r-xp 00000000 b3:02 22814 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/client/libjvm.so > 76983000-7698b000 ---p 00583000 b3:02 22814 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/client/libjvm.so > 7698b000-769b7000 rw-p 00583000 b3:02 22814 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/client/libjvm.so > 769b7000-76dda000 rw-p 00000000 00:00 0 > 76dda000-76df7000 r-xp 00000000 b3:02 3170 > /lib/arm-linux-gnueabihf/libgcc_s.so.1 > 76df7000-76e06000 ---p 0001d000 b3:02 3170 > /lib/arm-linux-gnueabihf/libgcc_s.so.1 > 76e06000-76e07000 rw-p 0001c000 b3:02 3170 > /lib/arm-linux-gnueabihf/libgcc_s.so.1 > 76e07000-76f32000 r-xp 00000000 b3:02 3155 > /lib/arm-linux-gnueabihf/libc-2.19.so > 76f32000-76f42000 ---p 0012b000 b3:02 3155 > /lib/arm-linux-gnueabihf/libc-2.19.so > 76f42000-76f44000 r--p 0012b000 b3:02 3155 > /lib/arm-linux-gnueabihf/libc-2.19.so > 76f44000-76f45000 rw-p 0012d000 b3:02 3155 > /lib/arm-linux-gnueabihf/libc-2.19.so > 76f45000-76f48000 rw-p 00000000 00:00 0 > 76f48000-76f4a000 r-xp 00000000 b3:02 3164 > /lib/arm-linux-gnueabihf/libdl-2.19.so > 76f4a000-76f59000 ---p 00002000 b3:02 3164 > /lib/arm-linux-gnueabihf/libdl-2.19.so > 76f59000-76f5a000 r--p 00001000 b3:02 3164 > /lib/arm-linux-gnueabihf/libdl-2.19.so > 76f5a000-76f5b000 rw-p 00002000 b3:02 3164 > /lib/arm-linux-gnueabihf/libdl-2.19.so > 76f5b000-76f6f000 r-xp 00000000 b3:02 22815 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/jli/libjli.so > 76f6f000-76f77000 ---p 00014000 b3:02 22815 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/jli/libjli.so > 76f77000-76f78000 rw-p 00014000 b3:02 22815 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/jli/libjli.so > 76f78000-76f8c000 r-xp 00000000 b3:02 3220 > /lib/arm-linux-gnueabihf/libpthread-2.19.so > 76f8c000-76f9c000 ---p 00014000 b3:02 3220 > /lib/arm-linux-gnueabihf/libpthread-2.19.so > 76f9c000-76f9d000 r--p 00014000 b3:02 3220 > /lib/arm-linux-gnueabihf/libpthread-2.19.so > 76f9d000-76f9e000 rw-p 00015000 b3:02 3220 > /lib/arm-linux-gnueabihf/libpthread-2.19.so > 76f9e000-76fa0000 rw-p 00000000 00:00 0 > 76fa1000-76faa000 r-xp 00000000 b3:02 22811 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libverify.so > 76faa000-76fb2000 ---p 00009000 b3:02 22811 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libverify.so > 76fb2000-76fb3000 rw-p 00009000 b3:02 22811 > /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/lib/arm/libverify.so > 76fb3000-76fb8000 r-xp 00000000 b3:02 20453 > /usr/lib/arm-linux-gnueabihf/libarmmem.so > 76fb8000-76fc7000 ---p 00005000 b3:02 20453 > /usr/lib/arm-linux-gnueabihf/libarmmem.so > 76fc7000-76fc8000 rw-p 00004000 b3:02 20453 > /usr/lib/arm-linux-gnueabihf/libarmmem.so > 76fc8000-76fe8000 r-xp 00000000 b3:02 3119 > /lib/arm-linux-gnueabihf/ld-2.19.so > 76fe9000-76ff1000 rw-s 00000000 b3:02 409006 /tmp/hsperfdata_root/2179 > 76ff1000-76ff2000 r--p 00000000 00:00 0 > 76ff2000-76ff5000 rw-p 00000000 00:00 0 > 76ff5000-76ff7000 rw-p 00000000 00:00 0 > 76ff7000-76ff8000 r--p 0001f000 b3:02 3119 > /lib/arm-linux-gnueabihf/ld-2.19.so > 76ff8000-76ff9000 rw-p 00020000 b3:02 3119 > /lib/arm-linux-gnueabihf/ld-2.19.so > 7ec8f000-7ecb0000 rwxp 00000000 00:00 0 [stack] > 7ed85000-7ed86000 r-xp 00000000 00:00 0 [sigpage] > 7ed86000-7ed87000 r--p 00000000 00:00 0 [vvar] > 7ed87000-7ed88000 r-xp 00000000 00:00 0 [vdso] > ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors] > VM Arguments: > jvm_args: -Dscala.usejavacp=true -Xms1g -Xmx1g > java_command: org.apache.spark.deploy.SparkSubmit --class > org.apache.spark.repl.Main --name Spark shell spark-shell > java_class_path (initial): > /opt/spark-1.6.1-bin-hadoop2.6/conf/:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar > Launcher Type: SUN_STANDARD > Environment Variables: > JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > USERNAME=root > SHELL=/bin/bash > DISPLAY=:0.0 > Signal Handlers: > SIGSEGV: [libjvm.so+0x506638], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGBUS: [libjvm.so+0x506638], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGFPE: [libjvm.so+0x3fd4ec], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGPIPE: [libjvm.so+0x3fd4ec], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGXFSZ: [libjvm.so+0x3fd4ec], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGILL: [libjvm.so+0x3fd4ec], sa_mask[0]=11111111011111111101111111111110, > sa_flags=SA_RESTART|SA_SIGINFO > SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none > SIGUSR2: [libjvm.so+0x3fd650], sa_mask[0]=00000000000000000000000000000000, > sa_flags=SA_RESTART|SA_SIGINFO > SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none > SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none > SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none > SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none > --------------- S Y S T E M --------------- > OS:PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)" > NAME="Raspbian GNU/Linux" > VERSION_ID="8" > VERSION="8 (jessie)" > ID=raspbian > ID_LIKE=debian > HOME_URL="http://www.raspbian.org/" > SUPPORT_URL="http://www.raspbian.org/RaspbianForums" > BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs" > uname:Linux 4.1.18-v7+ #846 SMP Thu Feb 25 14:22:53 GMT 2016 armv7l > libc:glibc 2.19 NPTL 2.19 > rlimit: STACK 8192k, CORE 0k, NPROC 7719, NOFILE 65536, AS infinity > load average:1.36 0.61 0.23 > /proc/meminfo: > MemTotal: 996776 kB > MemFree: 339060 kB > MemAvailable: 531380 kB > Buffers: 19752 kB > Cached: 197208 kB > SwapCached: 0 kB > Active: 500964 kB > Inactive: 119008 kB > Active(anon): 403400 kB > Inactive(anon): 7856 kB > Active(file): 97564 kB > Inactive(file): 111152 kB > Unevictable: 0 kB > Mlocked: 0 kB > SwapTotal: 102396 kB > SwapFree: 102396 kB > Dirty: 156 kB > Writeback: 0 kB > AnonPages: 402992 kB > Mapped: 77148 kB > Shmem: 8316 kB > Slab: 17608 kB > SReclaimable: 8168 kB > SUnreclaim: 9440 kB > KernelStack: 4304 kB > PageTables: 3788 kB > NFS_Unstable: 0 kB > Bounce: 0 kB > WritebackTmp: 0 kB > CommitLimit: 600784 kB > Committed_AS: 1410392 kB > VmallocTotal: 1056768 kB > VmallocUsed: 5176 kB > VmallocChunk: 806504 kB > CmaTotal: 8192 kB > CmaFree: 3732 kB > CPU:total 4 (ARMv7), vfp > /proc/cpuinfo: > processor : 0 > model name : ARMv7 Processor rev 4 (v7l) > BogoMIPS : 76.80 > Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt > vfpd32 lpae evtstrm crc32 > CPU implementer : 0x41 > CPU architecture: 7 > CPU variant : 0x0 > CPU part : 0xd03 > CPU revision : 4 > processor : 1 > model name : ARMv7 Processor rev 4 (v7l) > BogoMIPS : 76.80 > Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt > vfpd32 lpae evtstrm crc32 > CPU implementer : 0x41 > CPU architecture: 7 > CPU variant : 0x0 > CPU part : 0xd03 > CPU revision : 4 > processor : 2 > model name : ARMv7 Processor rev 4 (v7l) > BogoMIPS : 76.80 > Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt > vfpd32 lpae evtstrm crc32 > CPU implementer : 0x41 > CPU architecture: 7 > CPU variant : 0x0 > CPU part : 0xd03 > CPU revision : 4 > processor : 3 > model name : ARMv7 Processor rev 4 (v7l) > BogoMIPS : 76.80 > Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt > vfpd32 lpae evtstrm crc32 > CPU implementer : 0x41 > CPU architecture: 7 > CPU variant : 0x0 > CPU part : 0xd03 > CPU revision : 4 > Hardware : BCM2709 > Revision : a02082 > Serial : 0000000010852207 > Memory: 4k page, physical 996776k(339060k free), swap 102396k(102396k free) > vm_info: Java HotSpot(TM) Client VM (25.65-b01) for linux-arm-vfp-hflt JRE > (1.8.0_65-b17), built on Oct 6 2015 16:19:04 by "java_re" with gcc 4.7.2 > 20120910 (prerelease) > time: Wed Jun 8 22:49:10 2016 > elapsed time: 0 seconds (0d 0h 0m 0s) > Could you please help me resolve the issue?? -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org