Hi Peter,
Good Day!
I managed to build oozie 4.3.1 locally and tried to run job. Oozie starts up
fine. But I get below exception now in the oozie logs.
2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 -
SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-oozie-hado-W]
ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Start action
[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
with user-retry state : userRetryCount [0], userRetryMax [0],
userRetryInterval [10]
2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 -
SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-oozie-hado-W]
ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Error,
java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/io/Text;
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
I have checked for the jar file having this method and found that the class
' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present in the jar
file ' hadoop-yarn-common-2.4.0.jar' but could find the jar for '
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService'
, hence don’t know that which would be the dependent jar file here.
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ egrep -i
org.apache.hadoop.yarn.util.timeline.TimelineUtils *
Binary file hadoop-yarn-common-2.4.0.jar matches
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ egrep -i
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService
*
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ ls |wc -l
160
[hadoop@svdt5neonhadoop01 lib]$ pwd
/usr/local/oozie/lib
[hadoop@svdt5neonhadoop01 lib]$
Please help !
Thanks & Regards,
Jaboy Mathai
-----Original Message-----
From: Peter Cseh [mailto:gezap...@cloudera.com]
Sent: 01 June 2018 15:48
To: user@oozie.apache.org
Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
Subject: Re: Reg: Oozie 4.1.0 having jdbc error
I'm not aware of a binary distribution of Apache Oozie available to
download.
There are modified versions here and there (e.g. CDH's Oozie can be
installed via rpm:
https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_ig_cdh5_install.html
, but there are others available as well).
Peter
On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai <jaboy.mat...@flytxt.com>
wrote:
> Dear Peter,
>
> Thanks for your reply. I am using the same Maven 3.5.3 but the java
> version is '1.7.0_67'. Will try to build version 4.3.1 , but I am not
> sure because we don’t have local environment(having RHEL) with
> internet connection as you pointed. That’s why, I requested for a
> build, that may be readily available.
> Let me try building v.4.3.1 and will let you know.
>
> Best Regards
> Jaboy
>
> -----Original Message-----
> From: Peter Cseh [mailto:gezap...@cloudera.com]
> Sent: 01 June 2018 14:49
> To: user@oozie.apache.org
> Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> Garg
> Subject: Re: Reg: Oozie 4.1.0 having jdbc error
>
> Hi!
> Have you tried building 4.3.1?
> I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even
> cleaned my maven cache in between.
> I'm using Maven 3.5.3 with jdk1.8.
> To resolve the build issue you may compile Oozie locally and copy the
> distribution up to the server to see if it works that way. I suggest
> using Oozie 4.3.1 or 5.0.0.
>
> Based on the error messages during build (timeout) and Oozie startup
> (did not receive any packages) it looks like you're having issues with
> your network.
>
> hope it helps
> gp
>
> On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <jaboy.mat...@flytxt.com>
> wrote:
>
> > Hi,
> >
> >
> >
> > This is a bit urgent requirement for me, if someone has some idea ,
> > please help !
> >
> >
> >
> > BR,
> >
> > Jaboy
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mat...@flytxt.com]
> > *Sent:* 01 June 2018 10:37
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > Garg
> > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > + Looping Shivam.
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mat...@flytxt.com
> > <jaboy.mat...@flytxt.com>]
> > *Sent:* 31 May 2018 18:15
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> > *Subject:* Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > Hi Team,
> >
> >
> >
> > Need help.
> >
> >
> >
> > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same
> > network, we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we
> > had version 5.5.56 but we changed that, assuming compatibility issues).
> > Oozie gets started normally , but when I try to start to run the
> > workflow, I get jdbc error in oozie.log
> >
> >
> >
> > *Logs below:*
> >
> >
> >
> > Launcher ERROR, reason: Main class
> > [org.apache.oozie.action.hadoop.JavaMain],
> > main() threw exception, org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> > APP[SAFARICOM_KENYA_CBS_RECHARGE]
> > JOB[0000001-180531081106985-oozie-hado-W]
> > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_
> > Re charge_infile] Launcher exception: org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> >
> > at org.apache.oozie.action.hadoop.LauncherMain.run(
> > LauncherMain.java:39)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:57)
> >
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:606)
> >
> > at org.apache.oozie.action.hadoop.LauncherMapper.map(
> > LauncherMapper.java:226)
> >
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> >
> > at
> > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> >
> > at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >
> > at java.security.AccessController.doPrivileged(Native
> > Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1628)
> >
> > at
> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> >
> > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> > Could not get JDBC Connection; nested exception is
> > org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> >
> >
> > We have checked the port 3306 for mariadb using ‘nmap’ in linux and
> > have checked the connection hops from node1 to node5 (using
> > traceroute)and found that , there is only a single hop, that is,
> > node1 is directly accessing
> > node5 and vice-versa in the network(no switch IP shown intraceroute
> > output or switch firewalls expected).
> >
> >
> >
> > Checked for iptables and firewalls – no firewalls are running in the
> > cluster.
> >
> >
> >
> > *Please note:* Same job (workflow.xml) is running fine in one of our
> > other cluster with below versions:
> >
> >
> >
> > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
> >
> >
> >
> >
> >
> > *Attaching the logs having oozie startup and the job run log having
> > the error, with this e-mail.*
> >
> >
> >
> > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
> >
> >
> >
> > Tried to install oozie 4.3.0 but I am stuck as per below screenshot,
> > when I try to run below command:
> >
> >
> >
> > # ./mkdistro.sh –DskipTests
> >
> >
> >
> >
> >
> >
> >
> > I am able to wget the apache-17.pom from the linux shell prompt but
> > the build process fails. We have added the required proxy to access
> > the internet. I don’t know if any proxy to be added so that the
> > build process comes to know about the internet availability through
> > proxy.
> >
> >
> >
> > Please suggest a solution with the shortest approach possible to
> > solve this. Will appreciate if someone have the oozie 4.3.0 for RHEL
> > 7.5 readily available and could share.
> >
> >
> >
> > Below is the oozie related settings:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
> >
> >
> >
> > Setting OOZIE_HOME: /usr/local/oozie
> >
> > Setting OOZIE_CONFIG: /usr/local/oozie/conf
> >
> > Sourcing: /usr/local/oozie/conf/oozie-env.sh
> >
> > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> >
> > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> > HTTP_PORT}/oozie"
> >
> > Setting OOZIE_CONFIG_FILE: oozie-site.xml
> >
> > Setting OOZIE_DATA: /usr/local/oozie/data
> >
> > Setting OOZIE_LOG: /usr/local/oozie/logs
> >
> > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
> >
> > Setting OOZIE_LOG4J_RELOAD: 10
> >
> > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
> >
> > Setting OOZIE_HTTP_PORT: 11000
> >
> > Setting OOZIE_ADMIN_PORT: 11001
> >
> > Setting OOZIE_HTTPS_PORT: 11443
> >
> > Using OOZIE_BASE_URL: http://:/oozie
> >
> > Setting CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
> >
> > Setting OOZIE_HTTPS_KEYSTORE_PASS: password
> >
> > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
> >
> > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
> >
> > Setting CATALINA_PID: /usr/local/oozie/oozie-server/
> temp/oozie.pid
> >
> >
> >
> > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> > usr/local/oozie/logs/derby.log
> >
> > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> > -Doozie.config.dir=/usr/local/oozie/conf
> > -Doozie.log.dir=/usr/local/oozie/logs
> > -Doozie.data.dir=/usr/local/oozie/data
> > -Doozie.instance.id=svdt5neonh adoop01.safaricom.net
> > -Doozie.config.file=oozie-site.xml
> > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> > -Doozie.admin.port=11001 -Doozie.http.port=11000
> > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> > -Doozie.https.keystore.file=/home/hadoop/.keystore
> > -Doozie.https.keystore.pass=password -Djava.library.path=
> >
> > Using CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_HOME: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
> >
> > Using JRE_HOME: /usr/java/jdk1.7.0_67
> >
> > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
> >
> > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
> >
> >
> >
> >
> >
> > Thanks & Regards,
> >
> > *Jaboy Mathai*
> >
> >
> >
>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
