W Hasnaoui created AIRFLOW-4584:
-----------------------------------
Summary: Error when using ssh operateur to execute a sh script
from an remote server
Key: AIRFLOW-4584
URL: https://issues.apache.org/jira/browse/AIRFLOW-4584
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.10.2
Reporter: W Hasnaoui
hello guys;
i need yr help please, i'm new in apache airflow, and i'm trying to ssh
operateur to execute a shell script from a remote server, my code looks like
this:
t4 = SSHOperator(
ssh_conn_id='test_ssh',
task_id= 'Execute_transfert',
command="""sh 'scripts/jwi/test.sh'""",
dag=dag )
the only thing is inside my script (test.sh) i called a pentaho job (.kjb
extention), the line command gives:
LOGFILE="/xxx2/xxx3/logs/migxxx__`date "+%Y-%m-%d-%H%M"`.log"
JOBFILE="/xxx2/xxx3/xxx4/migxxx/avxxx.kjb"
PDI_LEVEL=Detailed
/folder1/folder2/kitchen.sh -file:$JOBFILE -level:$PDI_LEVEL -logfile:$LOGFILE
when running and afeter establishing connection to the remote server, the
execution faild, a snapshot of the log:
{{[2019-05-27 20:02:02,651] \{logging_mixin.py:95} INFO - [2019-05-27
20:02:02,651] \{transport.py:1746} INFO - Connected (version 2.0, client
OpenSSH_4.3) }}
{{[2019-05-27 20:02:05,877] \{logging_mixin.py:95} INFO - [2019-05-27
20:02:05,877] \{transport.py:1746} INFO - Authentication (publickey) failed. }}
{{[2019-05-27 20:02:05,897] \{logging_mixin.py:95} INFO - [2019-05-27
20:02:05,897] \{transport.py:1746} INFO - Authentication (password) successful!
}}
{{[2019-05-27 20:02:06,640] \{ssh_operator.py:133} INFO - INFO 27-05
18:22:07,371 - Using "/tmp/vfs_cache" as temporary files store. }}
{{[2019-05-27 20:02:06,777] \{models.py:1788} ERROR - SSH operator error:
'utf8' codec can't decode byte 0xe9 in position 63: invalid continuation byte }}
{{Traceback (most recent call last): }}
{{File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1657, in
_run_raw_task }}
{{result = task_copy.execute(context=context)}}
{{ File
"/usr/lib/python2.7/site-packages/airflow/contrib/operators/ssh_operator.py",
line 167, in execute}}
{{ raise AirflowException("SSH operator error: \{0}".format(str(e)))}}
{{ AirflowException: SSH operator error: 'utf8' codec can't decode byte 0xe9 in
position 63: invalid continuation byte }}
{{[2019-05-27 20:02:06,780] \{models.py:1817} INFO - All retries failed;
marking task as FAILED [2019-05-27 20:02:06,795] \{base_task_runner.py:101}
INFO - Job 1146: Subtask Execute_transfert Traceback (most recent call last): }}
{{[2019-05-27 20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/bin/airflow", line 32, in <module> [2019-05-27
20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert args.func(args) }}
{{[2019-05-27 20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py",
line 74, in wrapper [2019-05-27 20:02:06,797] \{base_task_runner.py:101} INFO -
Job 1146: Subtask Execute_transfert return f(*args, **kwargs) }}
{{[2019-05-27 20:02:06,797] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py",
line 526, in run [2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO -
Job 1146: Subtask Execute_transfert _run(args, dag, ti) }}
{{[2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py",
line 445, in _run [2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO -
Job 1146: Subtask Execute_transfert pool=args.pool, }}
{{[2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/utils/db.py",
line 73, in wrapper [2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO -
Job 1146: Subtask Execute_transfert return func(*args, **kwargs) }}
{{[2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/models.py",
line 1657, in _run_raw_task [2019-05-27 20:02:06,799]
\{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert result =
task_copy.execute(context=context) }}
{{[2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert File
"/usr/lib/python2.7/site-packages/airflow/contrib/operators/ssh_operator.py",
line 167, in execute [2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO
- Job 1146: Subtask Execute_transfert raise AirflowException("SSH operator
error: \{0}".format(str(e))) }}
{{[2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO - Job 1146: Subtask
Execute_transfert airflow.exceptions.AirflowException: SSH operator error:
'utf8' codec can't decode byte 0xe9 in position 63: invalid continuation byte}}
please help how can resolve this issue.
Many thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
