pankajastro opened a new pull request, #22772: URL: https://github.com/apache/airflow/pull/22772
Fail LocalFilesystemToGCSOperator if the src file does not exist `src` argument of LocalFilesystemToGCSOperator accept either list of source file path or a single source file path as a string. In the case of a single source file path we are using [glob](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L111) to parse the file path and glob return empty list if file path does not exist. In the next step, we iterate on this list and call [hook api](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L123) to update the file since the list is empty control is not going inside loop and task is succeeding even if the source file is not available. Change - Raise an exception if [filepath](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L111) list is empty After this change below task will fail if example-text.txt does not exist ``` upload_file = LocalFilesystemToGCSOperator( task_id="upload_file", src="example-text.txt", dst=DESTINATION_FILE_LOCATION, bucket=BUCKET_NAME, ) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
