RE: Reg: Oozie 4.1.0 having jdbc error
Dear Peter, There are 3 folders for the jar files in oozie home folder : 'lib' , 'libext' and 'libtools'. When I build oozie , I had Hadoop jar version '2.4.0' which I saw in 'lib' folder, but in 'libext' I had '2.6.0' version. Then I changed the jars in 'lib' folder to '2.6.0' to make the versions look similar in the oozie classpath. But that did not solve the issue. Hence the 'No such method Error' issue remains. Oozie has been compiled for RHEL 7.5 and Hadoop 2.6.0 . Best Regards Jaboy -Original Message- From: Peter Cseh [mailto:gezap...@cloudera.com] Sent: 11 June 2018 20:20 To: user@oozie.apache.org Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg Subject: Re: Reg: Oozie 4.1.0 having jdbc error Hey! Oozie 4.3.1 and newer releases are compiling with Hadoop 2.6.0. Why is the Hadoop jar you're referring is 2.4.0? Also, what Hadoop version you're compiling Oozie and you're code submitting to Oozie against? There might be different versions in play on the Oozie classpath and that can cause issues like this. gp On Thu, Jun 7, 2018 at 3:39 PM, Jaboy Mathai wrote: > Hi Peter, > > Good Day! > > I managed to build oozie 4.3.1 locally and tried to run job. Oozie > starts up fine. But I get below exception now in the oozie logs. > > 2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[003-180605235623225- > oozie-hado-W] > ACTION[003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Start action > [003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > with user-retry state : userRetryCount [0], userRetryMax [0], > userRetryInterval [10] > 2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[003-180605235623225- > oozie-hado-W] > ACTION[003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Error, > java.lang.NoSuchMethodError: > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService(Lorg/apache/hadoop/conf/ > Configuration;)Lorg/apache/hadoop/io/Text; > at > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl. > serviceInit(YarnClientImpl.java:166) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit( > ResourceMgrDelegate.java:102) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.( > ResourceMgrDelegate.java:96) > at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:112) > at > org.apache.hadoop.mapred.YarnClientProtocolProvider.create( > YarnClientProtocolProvider.java:34) > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) > at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82) > at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75) > > > I have checked for the jar file having this method and found that the > class ' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present > in the jar file ' hadoop-yarn-common-2.4.0.jar' but could find the jar > for ' > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService' > , hence don’t know that which would be the dependent jar file here. > > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils * Binary file > hadoop-yarn-common-2.4.0.jar matches > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService > * > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l > 160 > [hadoop@svdt5neonhadoop01 lib]$ pwd > /usr/local/oozie/lib > [hadoop@svdt5neonhadoop01 lib]$ > > > Please help ! > > Thanks & Regards, > Jaboy Mathai > > > > > -Original Message- > From: Peter Cseh [mailto:gezap...@cloudera.com] > Sent: 01 June 2018 15:48 > To: user@oozie.apache.org > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam > Garg > Subject: Re: Reg: Oozie 4.1.0 having jdbc error > > I'm not aware of a binary distribution of Apache Oozie available to > download. > There are modified versions here and there (e.g. CDH's Oozie can be > installed via rpm: > https://www.cloudera.com/documentation/enterprise/5-14- > x/topics/cdh_ig_cdh5_install.html > , but there are others available as well). > > Peter > > On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai > > wrote: > > > Dear Peter, > > > > Thanks for your reply. I am using the same Maven 3.5.3 but the java > > version is '1.7.0_67'. Will try to build version 4.3.1 , but I am > > not sure because we don’t have local environment(having RHEL)
Re: Reg: Oozie 4.1.0 having jdbc error
Hey! Oozie 4.3.1 and newer releases are compiling with Hadoop 2.6.0. Why is the Hadoop jar you're referring is 2.4.0? Also, what Hadoop version you're compiling Oozie and you're code submitting to Oozie against? There might be different versions in play on the Oozie classpath and that can cause issues like this. gp On Thu, Jun 7, 2018 at 3:39 PM, Jaboy Mathai wrote: > Hi Peter, > > Good Day! > > I managed to build oozie 4.3.1 locally and tried to run job. Oozie starts > up > fine. But I get below exception now in the oozie logs. > > 2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[003-180605235623225- > oozie-hado-W] > ACTION[003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Start action > [003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > with user-retry state : userRetryCount [0], userRetryMax [0], > userRetryInterval [10] > 2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[003-180605235623225- > oozie-hado-W] > ACTION[003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Error, > java.lang.NoSuchMethodError: > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService(Lorg/apache/hadoop/conf/ > Configuration;)Lorg/apache/hadoop/io/Text; > at > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl. > serviceInit(YarnClientImpl.java:166) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit( > ResourceMgrDelegate.java:102) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.( > ResourceMgrDelegate.java:96) > at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:112) > at > org.apache.hadoop.mapred.YarnClientProtocolProvider.create( > YarnClientProtocolProvider.java:34) > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) > at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82) > at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75) > > > I have checked for the jar file having this method and found that the > class > ' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present in the > jar > file ' hadoop-yarn-common-2.4.0.jar' but could find the jar for ' > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService' > , hence don’t know that which would be the dependent jar file here. > > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils * > Binary file hadoop-yarn-common-2.4.0.jar matches > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService > * > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l > 160 > [hadoop@svdt5neonhadoop01 lib]$ pwd > /usr/local/oozie/lib > [hadoop@svdt5neonhadoop01 lib]$ > > > Please help ! > > Thanks & Regards, > Jaboy Mathai > > > > > -Original Message- > From: Peter Cseh [mailto:gezap...@cloudera.com] > Sent: 01 June 2018 15:48 > To: user@oozie.apache.org > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg > Subject: Re: Reg: Oozie 4.1.0 having jdbc error > > I'm not aware of a binary distribution of Apache Oozie available to > download. > There are modified versions here and there (e.g. CDH's Oozie can be > installed via rpm: > https://www.cloudera.com/documentation/enterprise/5-14- > x/topics/cdh_ig_cdh5_install.html > , but there are others available as well). > > Peter > > On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai > wrote: > > > Dear Peter, > > > > Thanks for your reply. I am using the same Maven 3.5.3 but the java > > version is '1.7.0_67'. Will try to build version 4.3.1 , but I am not > > sure because we don’t have local environment(having RHEL) with > > internet connection as you pointed. That’s why, I requested for a > > build, that may be readily available. > > Let me try building v.4.3.1 and will let you know. > > > > Best Regards > > Jaboy > > > > -Original Message- > > From: Peter Cseh [mailto:gezap...@cloudera.com] > > Sent: 01 June 2018 14:49 > > To: user@oozie.apache.org > > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam > > Garg > > Subject: Re: Reg: Oozie 4.1.0 having jdbc error > > > > Hi! > > Have you tried building 4.3.1? > > I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even > > cleaned my maven cache in between. > > I'm using Maven 3.5.3 with jdk1.8. > > To resolve the build issue you may compile Oozie locally and copy
