[
https://issues.apache.org/jira/browse/OOZIE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ywheel updated OOZIE-2576:
--------------------------
Description:
We used oozie ssh action in a production environment, the following xml codes
is the example:
<workflow-app name="TestSsh" xmlns="uri:oozie:workflow:0.4">
<start to="ShellAction"/>
<action name="ShellAction">
<ssh xmlns="uri:oozie:ssh-action:0.1">
<host>[email protected]</host>
<command>sh /tmp/test.sh</command>
<capture-output/>
</ssh>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
However, we meet the errors below:
```
2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy ssh
base scripts to remote host [[email protected]]
2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error while executing
ssh EXECUTION
2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error starting action
[ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
[UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
program "scp": error=2, No such file or directory
at
org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
at
org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Cannot run program "scp": error=2, No such file
or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at
org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
at
org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
at
org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
at
org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
at
org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
... 10 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 17 more
```
I checked the codes in org.apache.oozie.action.ssh.SshActionExecutor class, and
I found oozie would execute 'scp' command to copy two
files(ssh-base.sh,ssh-wrapper.sh) to the workspace folder on remote host. But
the logs could not show any information about which file is not found.
The workspace folder on remote host was created so the 'ssh'+ 'mkdir' command
was executed successfully before 'scp', which means the authentication from
oozie server to remote host is correct.
And I tried the following command to scp that two shell files manually:
scp -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o
StrictHostKeyChecking=no -o ConnectTimeout=20
/var/run/cloudera-scm-agent/process/3518-oozie-OOZIE_SERVER/temp/oozie-oozi2411540976346867728.dir/ssh/ssh-base.sh
/var/run/cloudera-scm-agent/process/-oozie-OOZIE_SERVER/temp/oozie-oozi4864889223161337030.dir/ssh/ssh-wrapper.sh
root@remotehost:oozie-oozi/0000201-160113124428061-oozie-oozi-W/ShellAction--ssh/
It works by manually but why oozie server could not run 'scp'?
was:
We used oozie ssh action in a production environment, the following xml codes
is the example:
<workflow-app name="TestSsh" xmlns="uri:oozie:workflow:0.4">
<start to="ShellAction"/>
<action name="ShellAction">
<ssh xmlns="uri:oozie:ssh-action:0.1">
<host>[email protected]</host>
<command>sh /tmp/test.sh</command>
<capture-output/>
</ssh>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
However, we meet the errors below:
```
2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy ssh
base scripts to remote host [[email protected]]
2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error while executing
ssh EXECUTION
2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
JOB[0000201-160113124428061-oozie-oozi-W]
ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error starting action
[ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
[UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
program "scp": error=2, No such file or directory
at
org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
at
org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Cannot run program "scp": error=2, No such file
or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at
org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
at
org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
at
org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
at
org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
at
org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
... 10 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 17 more
```
I checked the codes in org.apache.oozie.action.ssh.SshActionExecutor class, and
I found oozie would execute 'scp' command to copy two
files(ssh-base.sh,ssh-wrapper.sh) to the workspace folder on remote host. But
the logs could not show any information about which file is not found.
The workspace folder on remote host was created so the 'ssh'+ 'mkdir' command
was executed successfully before 'scp', which means the authentication from
oozie server to remote host is correct.
And I tried the following command to scp that two shell files manually:
```
scp -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o
StrictHostKeyChecking=no -o ConnectTimeout=20
/var/run/cloudera-scm-agent/process/3518-oozie-OOZIE_SERVER/temp/oozie-oozi2411540976346867728.dir/ssh/ssh-base.sh
/var/run/cloudera-scm-agent/process/-oozie-OOZIE_SERVER/temp/oozie-oozi4864889223161337030.dir/ssh/ssh-wrapper.sh
root@remotehost:oozie-oozi/0000201-160113124428061-oozie-oozi-W/ShellAction--ssh/
```
It works! But why oozie server could not run 'scp'?
> Oozie ssh action Cannot run program "scp"
> -----------------------------------------
>
> Key: OOZIE-2576
> URL: https://issues.apache.org/jira/browse/OOZIE-2576
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: 4.0.0
> Environment: Oozie server build version: 4.0.0-cdh5.2.0
> Reporter: ywheel
> Priority: Minor
>
> We used oozie ssh action in a production environment, the following xml codes
> is the example:
> <workflow-app name="TestSsh" xmlns="uri:oozie:workflow:0.4">
> <start to="ShellAction"/>
> <action name="ShellAction">
> <ssh xmlns="uri:oozie:ssh-action:0.1">
> <host>[email protected]</host>
> <command>sh /tmp/test.sh</command>
> <capture-output/>
> </ssh>
> <ok to="end"/>
> <error to="kill"/>
> </action>
> <kill name="kill">
> <message>Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
> </workflow-app>
> However, we meet the errors below:
> ```
> 2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[0000201-160113124428061-oozie-oozi-W]
> ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy
> ssh base scripts to remote host [[email protected]]
> 2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[0000201-160113124428061-oozie-oozi-W]
> ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error while
> executing ssh EXECUTION
> 2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[0000201-160113124428061-oozie-oozi-W]
> ACTION[0000201-160113124428061-oozie-oozi-W@ShellAction] Error starting
> action [ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
> [UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
> org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
> program "scp": error=2, No such file or directory
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:281)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Cannot run program "scp": error=2, No such
> file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:485)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
> ... 10 more
> Caused by: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
> ... 17 more
> ```
> I checked the codes in org.apache.oozie.action.ssh.SshActionExecutor class,
> and I found oozie would execute 'scp' command to copy two
> files(ssh-base.sh,ssh-wrapper.sh) to the workspace folder on remote host. But
> the logs could not show any information about which file is not found.
> The workspace folder on remote host was created so the 'ssh'+ 'mkdir' command
> was executed successfully before 'scp', which means the authentication from
> oozie server to remote host is correct.
> And I tried the following command to scp that two shell files manually:
> scp -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o
> StrictHostKeyChecking=no -o ConnectTimeout=20
> /var/run/cloudera-scm-agent/process/3518-oozie-OOZIE_SERVER/temp/oozie-oozi2411540976346867728.dir/ssh/ssh-base.sh
>
> /var/run/cloudera-scm-agent/process/-oozie-OOZIE_SERVER/temp/oozie-oozi4864889223161337030.dir/ssh/ssh-wrapper.sh
>
> root@remotehost:oozie-oozi/0000201-160113124428061-oozie-oozi-W/ShellAction--ssh/
> It works by manually but why oozie server could not run 'scp'?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
