Thank you. I compileted with the command: $CLASSPATH=$(ls $HIVE_HOME/lib/hive-serde-*.jar):$(ls $HIVE_HOME/lib/hive-exec-*.jar):$(ls $HADOOP_HOME/hadoop-core-*.jar) $javac -cp $CLASSPATH PutMerge.java $ls PutMerge.class PutMerge.class
Now I tried: $java -cp $($HADOOP_HOME/bin/hadoop classpath):. PutMerge java -cp $($HADOOP_HOME/bin/hadoop classpath):. PutMerge 14/07/28 16:33:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/07/28 16:33:40 INFO security.JniBasedUnixGroupsMapping: Bailing out since native library couldn't be loaded Exception in thread "main" java.lang.ExceptionInInitializerError at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1028) at org.apache.hadoop.security.Groups.<init>(Groups.java:48) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:137) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:251) at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:229) at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:307) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:536) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:519) at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1717) at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1710) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1578) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:111) at PutMerge.main(PutMerge.java:12) Caused by: java.lang.RuntimeException at org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:44) ... 18 more If I try: $CLASSPATH=$(ls $HIVE_HOME/lib/hive-serde-*.jar):$(ls $HIVE_HOME/lib/hive-exec-*.jar):$(ls $HADOOP_HOME/hadoop-core-*.jar) $java -cp $($CLASSPATH classpath):. PutMerge -bash: /usr/lib/hive/lib/hive-serde-0.7.1-cdh3u4.jar:/usr/lib/hive/lib/hive-exec-0.7.1-cdh3u4.jar:/usr/lib/hadoop/hadoop-core-0.20.2-cdh3u4.jar: No such file or directory Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Could not find the main class: PutMerge. Program will exit. On Monday, July 28, 2014 12:42 AM, Harsh J <[email protected]> wrote: Please run it in the same style. The binary 'java' accepts a -cp param too: java -cp $($HADOOP_HOME/bin/hadoop classpath):. PutMerge On Mon, Jul 28, 2014 at 11:21 AM, R J <[email protected]> wrote: > Thanks a lot! I could compile with the added classpath: > $javac -cp $($HADOOP_HOME/bin/hadoop classpath) PutMerge.java > The above created PutMerge.class file. > Now I try to run: > $java PutMerge > Exception in thread "main" java.lang.NoClassDefFoundError: PutMerge > Caused by: java.lang.ClassNotFoundException: PutMerge > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > Could not find the main class: PutMerge. Program will exit. > > I get the above error. > I tried: > $set CLASSPATH=/usr/lib/hadoop/bin/hadoop > $java PutMerge > > I still get the error. > > > > On Sunday, July 27, 2014 10:16 PM, Harsh J <[email protected]> wrote: > > > The javac program can only find import dependencies referenced in a > program if it is also supplied on the javac classpath. Setting > HADOOP_HOME alone will not magically do this. Have you set an > appropriate classpath? > > Try as below, perhaps: > > javac -cp $($HADOOP_HOME/bin/hadoop classpath) PutMerge.java > > Alternatively, consider using a modern build helper tool such as > Apache Maven for writing java applications, they make your work > easier. > > On Mon, Jul 28, 2014 at 6:16 AM, R J <[email protected]> wrote: >> Hi All, >> >> I am new to programming on hadoop. I tried to compile the following >> program >> (example program from a hadoop book) on my linix server where I have >> Haddop >> installed: >> I get the errors: >> $javac PutMerge.java >> PutMerge.java:2: package org.apache.hadoop.conf does not exist >> import org.apache.hadoop.conf.Configuration; >> ^ >> PutMerge.java:3: package org.apache.hadoop.fs does not exist >> import org.apache.hadoop.fs.FSDataInputStream; >> ^ >> PutMerge.java:4: package org.apache.hadoop.fs does not exist >> import org.apache.hadoop.fs.FSDataOutputStream; >> ^ >> PutMerge.java:5: package org.apache.hadoop.fs does not exist >> import org.apache.hadoop.fs.FileStatus; >> ^ >> PutMerge.java:6: package org.apache.hadoop.fs does not exist >> import org.apache.hadoop.fs.FileSystem; >> ^ >> PutMerge.java:7: package org.apache.hadoop.fs does not exist >> import org.apache.hadoop.fs.Path; >> >> I have $HADOOP_HOME set u: >> $echo $HADOOP_HOME >> /usr/lib/hadoop >> >> Could you please suggest how to compile this program? Thanks a lot. >> >> Shu >> >> >> ====PutMerge.java========= >> import java.io.IOException; >> import org.apache.hadoop.conf.Configuration; >> import org.apache.hadoop.fs.FSDataInputStream; >> import org.apache.hadoop.fs.FSDataOutputStream; >> import org.apache.hadoop.fs.FileStatus; >> import org.apache.hadoop.fs.FileSystem; >> import org.apache.hadoop.fs.Path; >> public class PutMerge { >> >> public static void main(String[] args) throws IOException { >> Configuration conf = new Configuration(); >> FileSystem hdfs = FileSystem.get(conf); >> FileSystem local = FileSystem.getLocal(conf); >> >> Path inputDir = new Path(args[0]); >> Path hdfsFile = new Path(args[1]); >> >> try { >> FileStatus[] inputFiles = local.listStatus(inputDir); >> FSDataOutputStream out = hdfs.create(hdfsFile); >> >> for (int i=0; i<inputFiles.length; i++) { >> System.out.println(inputFiles[i].getPath().getName()); >> FSDataInputStream in = local.open(inputFiles[i].getPath()); >> byte buffer[] = new byte[256]; >> int bytesRead = 0; >> while( (bytesRead = in.read(buffer)) > 0) { >> out.write(buffer, 0, bytesRead); >> } >> in.close(); >> } >> out.close(); >> } catch (IOException e) { >> e.printStackTrace(); > >> } >> } >> } >> ============= >> > > > > -- > Harsh J > > > -- Harsh J
