[ https://issues.apache.org/jira/browse/HDDS-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670800#comment-16670800 ]
Soumitra Sulav commented on HDDS-781: ------------------------------------- Above exceptions were resolved by added mentioned jar containing the class in *hadoop/share/hadoop/mapreduce/lib/* folder of the mapreduce.tar.gz : {code:java} org/apache/hadoop/fs/ozone/OzoneFileSystem - hadoop-ozone-filesystem-0.3.0-SNAPSHOT.jar org/apache/ratis/proto/RaftProtos$ReplicationLevel - ratis-proto-0.3.0-9b2d7b6-SNAPSHOT.jar org/apache/ratis/thirdparty/com/google/protobuf/ProtocolMessageEnum - ratis-thirdparty-0.1.0-SNAPSHOT.jar{code} > Ambari HDP NoClassDefFoundError for MR jobs > ------------------------------------------- > > Key: HDDS-781 > URL: https://issues.apache.org/jira/browse/HDDS-781 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Filesystem > Affects Versions: 0.3.0 > Reporter: Soumitra Sulav > Priority: Major > > HDP integrated with Ambari has a > _/*usr/hdp/<hdp-version>/hadoop/mapreduce.tar.gz*_ file containing all the > libraries needed for a MR job to run and is copied in the yarn containers at > time of execution. > As introducing ozone filesystem, relevant jars need to be packaged as part of > the tar, also the tar is placed as part of _yum install hadoop_ components > done by Ambari during cluster setup. > During an MR Job run, I faced below java.lang.*NoClassDefFoundError* > exceptions : > > {code:java} > org/apache/hadoop/fs/ozone/OzoneFileSystem > org/apache/ratis/proto/RaftProtos$ReplicationLevel > org/apache/ratis/thirdparty/com/google/protobuf/ProtocolMessageEnum > {code} > > Adding the relevant jar in the mentioned tar file resolves the exception. > > Complete stacktrace for one of the *NoClassDefFoundError* exception : > {code:java} > 2018-10-31 10:03:05,191 ERROR [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster > java.lang.NoClassDefFoundError: > org/apache/ratis/proto/RaftProtos$ReplicationLevel > at org.apache.hadoop.hdds.scm.ScmConfigKeys.<clinit>(ScmConfigKeys.java:64) > at org.apache.hadoop.ozone.OzoneConfigKeys.<clinit>(OzoneConfigKeys.java:221) > at org.apache.hadoop.ozone.client.OzoneBucket.<init>(OzoneBucket.java:116) > at > org.apache.hadoop.ozone.client.rpc.RpcClient.getBucketDetails(RpcClient.java:421) > at org.apache.hadoop.ozone.client.OzoneVolume.getBucket(OzoneVolume.java:214) > at > org.apache.hadoop.fs.ozone.OzoneFileSystem.initialize(OzoneFileSystem.java:127) > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3354) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) > at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3403) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3371) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:477) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361) > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.<init>(FileOutputCommitter.java:160) > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.<init>(FileOutputCommitter.java:116) > at > org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory.createFileOutputCommitter(PathOutputCommitterFactory.java:134) > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitterFactory.createOutputCommitter(FileOutputCommitterFactory.java:35) > at > org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.getOutputCommitter(FileOutputFormat.java:338) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$3.call(MRAppMaster.java:552) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$3.call(MRAppMaster.java:534) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1802) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:534) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:311) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1760) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757) > at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691) > Caused by: java.lang.ClassNotFoundException: > org.apache.ratis.proto.RaftProtos$ReplicationLevel > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 29 more > 2018-10-31 10:03:05,203 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting > with status 1: java.lang.NoClassDefFoundError: > org/apache/ratis/proto/RaftProtos$ReplicationLevel{code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org