Brad Cox created OOZIE-1516:
-------------------------------
Summary: Usability issues submitting jobs
Key: OOZIE-1516
URL: https://issues.apache.org/jira/browse/OOZIE-1516
Project: Oozie
Issue Type: Improvement
Components: examples
Environment: Cloudera CDH4 latest on Centos
Reporter: Brad Cox
Priority: Critical
J Harsh suggested filing a usability issue concerning a problem he helped me
resolve via email. Problem seems to boil down to differences between localhost
and localhost.localdomain that I don't understand well enough to explain here.
I've enclosed the email thread as documentation.
J Harsh...
I believe we seem to be doing a strict hostname check in Apache Oozie
and not a resolving check. Could you report an improvement on the
Apache Oozie JIRA (https://issues.apache.org/jira/browse/OOZIE)
upstream to perhaps improve this for usability?
On Sun, Sep 1, 2013 at 2:41 AM, Brad Cox <[email protected]> wrote:
Copied examples to /user/cloudera/examples and changed nameNode to
hdfs://localhost.localdomain in job.properties. Results look like a good
sign.
[cloudera@localhost Desktop]$ oozie job -oozie http://localhost:11000/oozie
-config examples/apps/map-reduce/job.properties -run
job: 0000000-130831130415538-oozie-oozi-W
But in oozie console, got a job fail because jobTracker's not in whitelist
so changed that to just localhost.localdomain by analogy.
[cloudera@localhost Desktop]$ oozie job -oozie http://localhost:11000/oozie
-config examples/apps/map-reduce/job.properties -run
job: 0000001-130831130415538-oozie-oozi-W
Console says that try succeeded. So thanks for the help!!
All I'm missing is an understanding of why cloudera comes outOfBox with some
services configured as localhost and others localhost.localdomain. I always
thought they were equivalent. But these are huge differences; success vs
fail.
On Aug 31, 2013, at 4:58 PM, Brad Cox <[email protected]> wrote:
You mean use localhost.localdomain instead of localhost? Or should I define
a unique name with hostname cmd, restart all, and use that instead?
[cloudera@localhost Desktop]$ hostname
localhost.localdomain
On Aug 31, 2013, at 4:54 PM, Harsh J <[email protected]> wrote:
The whitelist message is close to the solution. Do not use 'localhost'
in your job.properties, but replace it with whatever the command
'hostname' produces on your terminal, and I believe it should do the
trick.
On Sat, Aug 31, 2013 at 11:11 PM, Brad Cox <[email protected]> wrote:
That's what I tried in previous attempts. That causes this
oozie job -oozie http://localhost:11000/oozie -config
examples/apps/map-reduce/job.properties -run
Error: HTTP error code: 500 : Internal Server Error
/var/log/lochost.date.log contains
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
localhost:8020/user/cloudera/examples/apps/map-reduce
at java.net.URI.checkPath(URI.java:1788)
examples is where I put it as you said
[cloudera@localhost Desktop]$ hdfs dfs -ls /
Found 8 items
drwxr-xr-x - hdfs supergroup 0 2013-08-30 15:28 /accumulo
drwxr-xr-x - oozie supergroup 0 2013-08-31 04:52 /examples
drwxr-xr-x - hbase hbase 0 2013-08-31 07:22 /hbase
drwxr-xr-x - hdfs supergroup 0 2013-08-30 16:38 /share
drwxr-xr-x - solr solr 0 2013-07-17 00:03 /solr
drwxrwxrwx - hdfs supergroup 0 2013-08-30 16:32 /tmp
drwxr-xr-x - hdfs supergroup 0 2013-08-30 16:32 /user
drwxr-xr-x - hdfs supergroup 0 2013-07-17 00:03 /var
All this is after hacking nameNode=localhost:8020 (was hdfs://...) in
job.properties based on the Running the Examples section of the Oozie docs.
On reverting that change, the message changed to the problem I had before:
oozie job -oozie http://localhost:11000/oozie -config
examples/apps/map-reduce/job.properties -run
Error: E0901 : E0901: Namenode [localhost:8020] not allowed, not in Oozies
whitelist
Based on that msg, I'd already changed
oozie.service.HadoopAccessorService.nameNode.whitelist to localhost:8020 in
/etc/oozie/conf/oozie-site.xml and just made a similar change to
jobTracker.whitelist to localhost:8021. Best I've been able to do is switch
mystifying messages.
On Aug 31, 2013, at 12:22 PM, Harsh J <[email protected]> wrote:
Hi Brad,
In Oozie, an simple "application" consists of a directory on HDFS that
carries workflow.xml, and associated libraries/files.
To invoke an instance of such an application, we use a local
job.properties file that carries any variable parameter values, to
begin the application with.
Typically, to run the examples out of the box on a standalone cluster,
you can run the following set of commands (assuming you do have the
oozie-examples tarball):
$ tar xf oozie-examples.tar.gz
$ hadoop fs -put examples examples
$ oozie job -oozie http://localhost:11000/oozie -config
examples/apps/map-reduce/job.properties -run
Can you tell me what error you face if you try this out?
On Sat, Aug 31, 2013 at 8:22 PM, Brad Cox <[email protected]> wrote:
I'm trying the ozzie examples in CDH4.1 and the error messages make no
sense. Got through the others by editing the properties file but this one
really stumps me.
I've copied the ozzie examples folder to /home/cloudera/Desktop and have
verified that job.properties path is right.
[cloudera@localhost Desktop]$ whoami
cloudera
[cloudera@localhost Desktop]$ ls
/home/cloudera/Desktop/examples/apps/map-reduce/job.properties
Run as cloudera user and trying various permutations on the path
(with/without file:// prefix) I keep getting this.
oozie job -oozie http://localhost:11000/oozie -config
file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties -run
java.io.IOException: configuration file
[file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties] not
found
at org.apache.oozie.cli.OozieCLI.getConfiguration(OozieCLI.java:641)
at org.apache.oozie.cli.OozieCLI.jobCommand(OozieCLI.java:795)
at org.apache.oozie.cli.OozieCLI.processCommand(OozieCLI.java:512)
at org.apache.oozie.cli.OozieCLI.run(OozieCLI.java:485)
at org.apache.oozie.cli.OozieCLI.main(OozieCLI.java:182)
configuration file
[file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties] not
found
I also tried putting the examples in hdfs as the instructions said but that
doesn't help. In fact that's entirely the wrong idea according to some
emails I've seen.
Dr. Brad J. Cox Cell: 703-594-1883 Blog: http://bradjcox.blogspot.com
http://virtualschool.edu
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
