ReneTC opened a new issue, #28603:
URL: https://github.com/apache/airflow/issues/28603

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   I have an issue with the `SFTPOperator` from 
`airflow.providers.sftp.operators.sftp`.. Mostly it works perfectly.
   However, sometimes it refuses to download the complete file and is just 
stuck "downloading". A job has been running for 7 days to download a 38 mb 
file. I can see in the output file there is 37.5 mb out of the 38 mb. Last 
output of the log is `Starting to transfer from ftp_server_file.csv to 
local_gcs_file.csv`
   
   But most of the time it works perfectly - It has downalded a 39 mb file 
before without issues.
   
   I'm downloading from a sftp server to google cloud storage in Airflow 2.3.3.
   
   
   ### What you think should happen instead
   
   Somehow this operator still think it is downloading. Which makes sense since 
the file is not 100% downloaded. But the download speed must be close to 0 byte 
per second. Job should terminate. 
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Google Cloud Composer
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Composer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to