[
https://issues.apache.org/jira/browse/OOZIE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163060#comment-13163060
]
[email protected] commented on OOZIE-578:
-----------------------------------------------------
bq. On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
bq. >
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd,
line 34
bq. > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
bq. >
bq. > env vars setting is missing, this is quite important when using
shells.
bq. >
bq. > we should be able to specify multiple ENV vars, with one of the two
syntaxes:
bq. >
bq. > <env-var>VAR=VALUE</env-var>
bq. >
bq. > or
bq. >
bq. > <environment>
bq. > <variable>
bq. > <name>VALUE</name>
bq. > <value>VALUE</value>
bq. > </variable>
bq. > </environment>
bq. >
bq.
bq. Mohammad Islam wrote:
bq. I considered that also an option. So user can specify it here and
oozie will add it to oozie.launcher.mapred.child.env.
bq. I will add that.
bq. In addition, currently, User could still add the env directly through
oozie.launcher.mapred.child.env.
bq.
bq. Alejandro Abdelnur wrote:
bq. I don't think these ENV settings should affect the launcher ENV, these
ENV settings are exclusively for the SHELL it will be executed by the launcher.
bq.
bq. If the user wants to alter the launcher ENV for some reason, they
always can do it with the oozie.launcher.mapred.child.env, but by definition,
anything that is oozie.launcher.* is not propagated to the action itself.
My use case is as follows:
1. <file>myscr.sh</file>
2. <exec>myscr.sh<exec>
In this case, if Oozie LM executes the command, it will not find the executable
"myscr.sh" because PATH variable doesn't contain CWD(.).
However, if it is like <exec>./myscr.sh<exec>, it will work. That means user
has to give the full path of the executable. How can a user configure this
through PATH variable.
bq. On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
bq. >
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java,
line 106
bq. > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line106>
bq. >
bq. > as the shellmain is forking a new process, these changes are not
required for the launcher mapper process, but for the process the shellmain
launches.
bq.
bq. Mohammad Islam wrote:
bq. I think we need it here. I spend a lot of time on this.
bq. For example, my script is in CWD of LM. If I don't set it up in
*.child.env. It will never get the script if I do it RunTime.exec(args, envp).
bq. Other environment passed in RunTime.exec(args, envp), doesn't have any
impact on finding the executables own path. However, it impacts on within the
spawned script itself.
bq. That's why I do it in *.child.env that will be available to LM to find
the shell executable itself as well as to the spawned script.
bq.
bq.
bq. Alejandro Abdelnur wrote:
bq. If using the ProcessBuilder class, you can easily set the workding
directory with the directory(File) method.
Does it add the CWD in the classpath?
bq. On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
bq. >
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java,
line 132
bq. > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line132>
bq. >
bq. > consuming the stdout/stderr of the forked process must be done in
different threads, else you may be be blocking processing/overflowing buffers.
bq. >
bq. >
bq.
bq. Mohammad Islam wrote:
bq. Are you asking to spawn a new Thread to do it?
bq.
bq. Alejandro Abdelnur wrote:
bq. actually, 2 new threads, one to consume STDOUT and other STDERR, they
should be daemon threads and exit the loop on -1 read as well.
bq.
bq. both outputs should be redirected to to the launcher process
STDOUT/STDERR (as then they end up in the launcher logs).
bq.
bq. the STDOUT should also be captured (a la tee) so it can be used for
<capture-output> functionality.
Not sure about the capture-output. If user writes a lot of output for its own
debugging and we consider that as capture-output, it might not work or go
out-of-max-size error very often.
What we could do, by default, capture-output will be false.
If a user needs capture output, we should write it to the capture-output file.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3000/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-04 08:05:24)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. More context at JIRA OOZIE-578.
bq.
bq.
bq. This addresses bug OOZIE-578.
bq. https://issues.apache.org/jira/browse/OOZIE-578
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java
1209346
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
1209346
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
1209346
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki
1209346
bq.
bq. Diff: https://reviews.apache.org/r/3000/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Mohammad
bq.
bq.
> Support shell action in Oozie WF
> --------------------------------
>
> Key: OOZIE-578
> URL: https://issues.apache.org/jira/browse/OOZIE-578
> Project: Oozie
> Issue Type: New Feature
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
>
> Running a script (shell/perl/python/?) from a workflow is a very common
> use-case. Currently, Oozie user needs to write a custom java code to execute
> the script which is not convenient.
> This JIRA is to discuss the feasibility and scope of the feature. The
> proposed design of this feature will also be included.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
