On 02/12/2015 05:15 PM, Trevor McKay wrote:
Hi folks,
Here is another way to do this. Lu had mentioned Oozie shell actions
previously.
Sahara doesn't support them, but I played with it from the Oozie command
line
to verify that it solves our hbase problem, too.
We can potentially create a blueprint to build a simple Shell action
around a
user-supplied script and supporting files. The script and files would
all be stored
in Sahara as job binaries (Swift or internal db) and referenced the same
way. The exec
target must be on the path at runtime, or included in the working dir.
To do this, I simply put workflow.xml, doit.sh, and the test jar into
a directory in hdfs. Then I ran it with the Oozie cli using the job.xml
config file
configured to point at the hdfs dir. Nothing special here, just
standard Oozie
job execution.
very cool Trevor, i wonder if there is a greater pattern here we could
identify. namely, the idea that a user could upload multiple job
binaries and create some sort of nesting structure to the way they are
interpreted. maybe there could be a standard substitution method for
script like files. in this respect a user could create a standard
wrapper script and allow different binary names to be substituted into
the script. this may be too complicated but it occurred to me while
reading your results.
regardless of the greater pattern, this is a good window into more ways
for us to control the command arguments.
mike
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
