Hi again, previously I had some configuration problems in running oozie 4.2.0 against hadoop 2.7.0 in localmode but eventually I could make it working. I am able to execute all my use cases in localmode. Now I'm deploying my system on a EC2 cluster. I'm using the same distro of oozie that I built in localmode since I am using exaclty the same components versions. I'm using just two node at the moment: in the master node I have namenode, resourcemanager and jobhistoryserver daemons while on the slave I have datanode and nodemanager daemons correctly running (I can interact with the hdfs and running map-reduce example jobs). I installed the oozie distro on the master node and I am using the same user for using either hadoop and oozie ( as I'm doing in local mode)
I have two main issues: 1) Oozie server seems up and running, I can execute jobs through it. The jobs are actually executed and completed by yarn but on the oozie console they stay in RUNNING mode and only after 10 minutes they switch to SUCCEEDED or FAILED according to the result. Checking at the syslog of each job I can see two suspicious lines: WARN [uber-SubtaskRunner] org.apache.hadoop.mapred.LocalContainerLauncher: Unable to delete unexpected local file/dir .action.xml.crc: insufficient permissions? ... ERROR [uber-EventHandler] org.apache.hadoop.mapred.LocalContainerLauncher: Returning, interrupted : java.lang.InterruptedException The rest of the log seems normal: I can follow the execution of the job from the submission till the completion. 2) I had this problem also in localmode: when trying to execute a spark action specifying the master=yarn-client in the job.properties file I get this error in the stderr log file: Error: Could not load YARN classes. This copy of Spark may not have been compiled with YARN support. The error is clear, but I'm using the share libs that were shipped with the latest oozie version, which is supposed to be able to execute spark action on yarn. Am I missing some additional libs? I report some configuration files that I'm using example of job.property: nameNode=hdfs://172.31.25.237:9000 (internal IP address of my EC2 master node) jobTracker=http://172.31.25.237:8032 queueName=default examplesRoot=examples oozie.wf.application.path=${nameNode}/user/${user.name }/${examplesRoot}/apps/java-main oozie-site.xml <property> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> <value>*=/opt/hadoop-2.7.0/hadoop-2.7.0/etc/hadoop</value> </property> <property> <name>oozie.processing.timezone</name> <value>GMT+0200</value> </property> <property> <name>oozie.base.url</name> <value>http://publicEC2ip:11000/oozie/</value> </property> -- Matteo Remo Luzzi
