Apache Bigtop detects JVM based on the most recent version first, if you use packages based on their framework: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/bigtop-utils/bigtop-detect-javahome (switch branch/tag appropriately to a specific release you may be using/based off)
Within Apache Hadoop, we rely on the definition of JAVA_HOME explicitly. In a packaged environment, the above script typically supplies it. On Sat, Jul 19, 2014 at 6:35 AM, Chris Mawata <[email protected]> wrote: > Great that you got it sorted out. I'm afraid I don't know if there is a > configuration that would automatically check the versions -- maybe someone > who knows might chime in. > Cheers > Chris > > On Jul 18, 2014 3:06 PM, "andrew touchet" <[email protected]> wrote: >> >> Thanks Chris! >> >> The issue was that even though I set jdk-7u21 as my default, it checked >> for /usr/java/jdk-1.6* first. Even though it was compiled with 1.7. >> >> Is there anyway to generate a proper hadoop-config.sh to reflect the minor >> version hadoop was built with? So that in my case, it would check for >> /usr/java/jdk-1.7* instead? I appreciate the help! >> >> >> On Thu, Jul 17, 2014 at 11:11 PM, Chris Mawata <[email protected]> >> wrote: >>> >>> Yet another place to check -- in the hadoop-env.sh file there is also a >>> JAVA_HOME setting. >>> Chris >>> >>> On Jul 17, 2014 9:46 PM, "andrew touchet" <[email protected]> wrote: >>>> >>>> Hi Fireflyhoo, >>>> >>>> Below I follow the symbolic links for the jdk-7u21. These links are >>>> changed accordingly as I change between versions. Also, I have 8 datanodes >>>> and 2 other various servers that are capable of mounting /hdfs. So it is >>>> just this server is an issue. >>>> >>>> $ java -version >>>> java version "1.7.0_21" >>>> Java(TM) SE Runtime Environment (build 1.7.0_21-b11) >>>> Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode) >>>> >>>> java >>>> $ ls -l `which java` >>>> lrwxrwxrwx 1 root root 26 Jul 17 19:50 /usr/bin/java -> >>>> /usr/java/default/bin/java >>>> $ ls -l /usr/java/default >>>> lrwxrwxrwx 1 root root 16 Jul 17 19:50 /usr/java/default -> >>>> /usr/java/latest >>>> $ ls -l /usr/java/latest >>>> lrwxrwxrwx 1 root root 21 Jul 17 20:29 /usr/java/latest -> >>>> /usr/java/jdk1.7.0_21 >>>> >>>> jar >>>> $ ls -l `which jar` >>>> lrwxrwxrwx 1 root root 21 Jul 17 20:18 /usr/bin/jar -> >>>> /etc/alternatives/jar >>>> $ ls -l /etc/alternatives/jar >>>> lrwxrwxrwx 1 root root 29 Jul 17 20:26 /etc/alternatives/jar -> >>>> /usr/java/jdk1.7.0_21/bin/jar >>>> >>>> javac >>>> $ ls -l `which javac` >>>> lrwxrwxrwx 1 root root 23 Jul 17 20:18 /usr/bin/javac -> >>>> /etc/alternatives/javac >>>> $ ls -l /etc/alternatives/javac >>>> lrwxrwxrwx 1 root root 31 Jul 17 20:26 /etc/alternatives/javac -> >>>> /usr/java/jdk1.7.0_21/bin/javac >>>> >>>> Now that I've tried version from 6 & 7, I'm really not sure what is >>>> causing this issue. >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Jul 17, 2014 at 8:21 PM, [email protected] >>>> <[email protected]> wrote: >>>>> >>>>> I think you first confirm you local java version , >>>>> Some liux will pre-installed java ,that version is very low >>>>> >>>>> ________________________________ >>>>> [email protected] >>>>> >>>>> >>>>> From: andrew touchet >>>>> Date: 2014-07-18 09:06 >>>>> To: user >>>>> Subject: Re: HDFS input/output error - fuse mount >>>>> Hi Chris, >>>>> >>>>> I tried to mount /hdfs with java versions below but there was no change >>>>> in output. >>>>> jre-7u21 >>>>> jdk-7u21 >>>>> jdk-7u55 >>>>> jdk1.6.0_31 >>>>> jdk1.6.0_45 >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Jul 17, 2014 at 6:56 PM, Chris Mawata <[email protected]> >>>>> wrote: >>>>>> >>>>>> Version 51 ia Java 7 >>>>>> Chris >>>>>> >>>>>> On Jul 17, 2014 7:50 PM, "andrew touchet" <[email protected]> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Hadoop package installed: >>>>>>> hadoop-0.20-0.20.2+737-33.osg.el5.noarch >>>>>>> >>>>>>> Operating System: >>>>>>> CentOS release 5.8 (Final) >>>>>>> >>>>>>> I am mounting HDFS from my namenode to another node with fuse. After >>>>>>> mounting to /hdfs, any attempts to 'ls', 'cd', or use 'hadoop fs' leads >>>>>>> to >>>>>>> the below output. >>>>>>> >>>>>>> >>>>>>> $ls /hdfs >>>>>>> ls: /hdfs: Input/output error >>>>>>> $hadoop fs -ls >>>>>>> Exception in thread "main" java.lang.UnsupportedClassVersionError: >>>>>>> org/apache/hadoop/fs/FsShell : Unsupported major.minor version 51.0 >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >>>>>>> at >>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >>>>>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >>>>>>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>>>> Could not find the main class: org.apache.hadoop.fs.FsShell. Program >>>>>>> will exit. >>>>>>> >>>>>>> >>>>>>> I have attempted to mount /hdfs manually in debug mode and then >>>>>>> attempted to access /hdfs from a different terminal. This is the >>>>>>> output. The >>>>>>> namenode is glados. The server where /hdfs is being mounted is glados2. >>>>>>> >>>>>>> >>>>>>> $hdfs -oserver=glados,port=9000,rdbuffer=131072,allow_other /hdfs -d >>>>>>> fuse-dfs ignoring option allow_other >>>>>>> ERROR fuse_options.c:162 fuse-dfs didn't recognize /hdfs,-2 >>>>>>> >>>>>>> fuse-dfs ignoring option -d >>>>>>> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56 >>>>>>> INIT: 7.10 >>>>>>> flags=0x0000000b >>>>>>> max_readahead=0x00020000 >>>>>>> INFO fuse_init.c:115 Mounting glados:9000 >>>>>>> Exception in thread "main" java.lang.UnsupportedClassVersionError: >>>>>>> org/apache/hadoop/conf/Configuration : Unsupported major.minor version >>>>>>> 51.0 >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >>>>>>> at >>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >>>>>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >>>>>>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>>>> Can't construct instance of class >>>>>>> org.apache.hadoop.conf.Configuration >>>>>>> ERROR fuse_init.c:127 Unable to establish test connection to server >>>>>>> INIT: 7.8 >>>>>>> flags=0x00000001 >>>>>>> max_readahead=0x00020000 >>>>>>> max_write=0x00020000 >>>>>>> unique: 1, error: 0 (Success), outsize: 40 >>>>>>> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56 >>>>>>> Exception in thread "Thread-0" >>>>>>> java.lang.UnsupportedClassVersionError: >>>>>>> org/apache/hadoop/conf/Configuration >>>>>>> : Unsupported major.minor version 51.0 >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >>>>>>> at >>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >>>>>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >>>>>>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>>>> Can't construct instance of class >>>>>>> org.apache.hadoop.conf.Configuration >>>>>>> ERROR fuse_connect.c:83 Unable to instantiate a filesystem for >>>>>>> user027 >>>>>>> ERROR fuse_impls_getattr.c:40 Could not connect to glados:9000 >>>>>>> unique: 2, error: -5 (Input/output error), outsize: 16 >>>>>>> unique: 3, opcode: GETATTR (3), nodeid: 1, insize: 56 >>>>>>> >>>>>>> I adopted this system after this was already setup, so I do not know >>>>>>> which java version was used during install. Currently I'm using: >>>>>>> >>>>>>> $java -version >>>>>>> java version "1.6.0_45" >>>>>>> Java(TM) SE Runtime Environment (build 1.6.0_45-b06) >>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) >>>>>>> >>>>>>> $java -version >>>>>>> java version "1.6.0_45" >>>>>>> Java(TM) SE Runtime Environment (build 1.6.0_45-b06) >>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode) >>>>>>> >>>>>>> >>>>>>> Is my java version really the cause of this issue? What is the >>>>>>> correct java version to be used for this version of hadoop. I have also >>>>>>> tried 1.6.0_31 but no changes were seen. >>>>>>> >>>>>>> If java isn't my issue, then what is? >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Andrew >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >> > -- Harsh J
