[
https://issues.apache.org/jira/browse/AIRFLOW-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jack updated AIRFLOW-4382:
--------------------------
Description:
The SFTPOperator tests seems to fail many times.
Examples for PRs where this test fails:
[https://travis-ci.org/apache/airflow/jobs/522789194]
[https://travis-ci.org/apache/airflow/jobs/522931911]
{code:java}
INFO [airflow.task] Dependencies all met for <TaskInstance:
unit_teststest_schedule_dag_once.test_sftp 2019-04-22 06:02:24.255945+00:00
[None]> INFO [airflow.task]
--------------------------------------------------------------------------------
INFO [airflow.task] Starting attempt 1 of 1 INFO [airflow.task]
--------------------------------------------------------------------------------
INFO [airflow.task] Executing <Task(SFTPOperator): test_sftp> on 2019-04-22
06:02:24.255945+00:00 INFO [paramiko.transport] Connected (version 2.0, client
OpenSSH_7.2p2) INFO [paramiko.transport] Authentication (publickey) successful!
INFO [paramiko.transport.sftp] [chan 0] Opened sftp connection (server version
3) INFO [airflow.task.operators] Starting to transfer from
/tmp/test_remote_file to /tmp/tmp2/test_local_file ERROR [airflow.task] Error
while transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file,
error: [Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'
Traceback (most recent call last): File
"/app/airflow/contrib/operators/sftp_operator.py", line 138, in execute
sftp_client.get(self.remote_filepath, self.local_filepath) File
"/app/.tox/py35-backend_sqlite-env_docker/lib/python3.5/site-packages/paramiko/sftp_client.py",
line 801, in get with open(localpath, "wb") as fl: FileNotFoundError: [Errno
2] No such file or directory: '/tmp/tmp2/test_local_file' During handling of
the above exception, another exception occurred: Traceback (most recent call
last): File "/app/airflow/models/taskinstance.py", line 888, in _run_raw_task
result = task_copy.execute(context=context) File
"/app/airflow/contrib/operators/sftp_operator.py", line 155, in execute
.format(file_msg, str(e))) airflow.exceptions.AirflowException: Error while
transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file, error:
[Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'{code}
was:
The SFTPOperator tests seems to fail many times.
{code:java}
INFO [airflow.task] Dependencies all met for <TaskInstance:
unit_teststest_schedule_dag_once.test_sftp 2019-04-22 06:02:24.255945+00:00
[None]> INFO [airflow.task]
--------------------------------------------------------------------------------
INFO [airflow.task] Starting attempt 1 of 1 INFO [airflow.task]
--------------------------------------------------------------------------------
INFO [airflow.task] Executing <Task(SFTPOperator): test_sftp> on 2019-04-22
06:02:24.255945+00:00 INFO [paramiko.transport] Connected (version 2.0, client
OpenSSH_7.2p2) INFO [paramiko.transport] Authentication (publickey) successful!
INFO [paramiko.transport.sftp] [chan 0] Opened sftp connection (server version
3) INFO [airflow.task.operators] Starting to transfer from
/tmp/test_remote_file to /tmp/tmp2/test_local_file ERROR [airflow.task] Error
while transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file,
error: [Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'
Traceback (most recent call last): File
"/app/airflow/contrib/operators/sftp_operator.py", line 138, in execute
sftp_client.get(self.remote_filepath, self.local_filepath) File
"/app/.tox/py35-backend_sqlite-env_docker/lib/python3.5/site-packages/paramiko/sftp_client.py",
line 801, in get with open(localpath, "wb") as fl: FileNotFoundError: [Errno
2] No such file or directory: '/tmp/tmp2/test_local_file' During handling of
the above exception, another exception occurred: Traceback (most recent call
last): File "/app/airflow/models/taskinstance.py", line 888, in _run_raw_task
result = task_copy.execute(context=context) File
"/app/airflow/contrib/operators/sftp_operator.py", line 155, in execute
.format(file_msg, str(e))) airflow.exceptions.AirflowException: Error while
transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file, error:
[Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'{code}
> SFTPOperator tests are flaky
> ----------------------------
>
> Key: AIRFLOW-4382
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4382
> Project: Apache Airflow
> Issue Type: Wish
> Components: tests
> Reporter: jack
> Priority: Major
> Fix For: 1.10.4
>
>
> The SFTPOperator tests seems to fail many times.
> Examples for PRs where this test fails:
> [https://travis-ci.org/apache/airflow/jobs/522789194]
> [https://travis-ci.org/apache/airflow/jobs/522931911]
>
> {code:java}
>
> INFO [airflow.task] Dependencies all met for <TaskInstance:
> unit_teststest_schedule_dag_once.test_sftp 2019-04-22 06:02:24.255945+00:00
> [None]> INFO [airflow.task]
> --------------------------------------------------------------------------------
> INFO [airflow.task] Starting attempt 1 of 1 INFO [airflow.task]
> --------------------------------------------------------------------------------
> INFO [airflow.task] Executing <Task(SFTPOperator): test_sftp> on 2019-04-22
> 06:02:24.255945+00:00 INFO [paramiko.transport] Connected (version 2.0,
> client OpenSSH_7.2p2) INFO [paramiko.transport] Authentication (publickey)
> successful! INFO [paramiko.transport.sftp] [chan 0] Opened sftp connection
> (server version 3) INFO [airflow.task.operators] Starting to transfer from
> /tmp/test_remote_file to /tmp/tmp2/test_local_file ERROR [airflow.task] Error
> while transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file,
> error: [Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'
> Traceback (most recent call last): File
> "/app/airflow/contrib/operators/sftp_operator.py", line 138, in execute
> sftp_client.get(self.remote_filepath, self.local_filepath) File
> "/app/.tox/py35-backend_sqlite-env_docker/lib/python3.5/site-packages/paramiko/sftp_client.py",
> line 801, in get with open(localpath, "wb") as fl: FileNotFoundError:
> [Errno 2] No such file or directory: '/tmp/tmp2/test_local_file' During
> handling of the above exception, another exception occurred: Traceback
> (most recent call last): File "/app/airflow/models/taskinstance.py", line
> 888, in _run_raw_task result = task_copy.execute(context=context) File
> "/app/airflow/contrib/operators/sftp_operator.py", line 155, in execute
> .format(file_msg, str(e))) airflow.exceptions.AirflowException: Error while
> transferring from /tmp/test_remote_file to /tmp/tmp2/test_local_file, error:
> [Errno 2] No such file or directory: '/tmp/tmp2/test_local_file'{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
