mahadirz opened a new issue #9780:
URL: https://github.com/apache/airflow/issues/9780
Some modules version that could help:
apache-airflow==1.10.10
google-cloud-storage==1.26.0
## Logs
```
[2020-07-12 08:03:03,455] {taskinstance.py:1145} ERROR - upload() got an
unexpected keyword argument 'bucket_name'
Traceback (most recent call last):
File
"/airflow/miniconda3/envs/airflow/lib/python3.6/site-packages/airflow/models/taskinstance.py",
line 983, in _run_raw_task
result = task_copy.execute(context=context)
File
"/airflow/miniconda3/envs/airflow/lib/python3.6/site-packages/airflow/contrib/operators/dataproc_operator.py",
line 1145, in execute
self.main = self._upload_file_temp(bucket, self.main)
File
"/airflow/miniconda3/envs/airflow/lib/python3.6/site-packages/airflow/contrib/operators/dataproc_operator.py",
line 1108, in _upload_file_temp
filename=local_file
TypeError: upload() got an unexpected keyword argument 'bucket_name'
[2020-07-12 08:03:03,457] {taskinstance.py:1202} INFO - Marking task as
FAILED.dag_id=snowplow_pipeline, task_id=text_to_partitioned_parquet,
execution_date=20200711T000000, start_date=20200712T000301,
end_date=20200712T000303
[2020-07-12 08:03:11,950] {logging_mixin.py:112} INFO - [2020-07-12
08:03:11,949] {local_task_job.py:103} INFO - Task exited with return code 1
```
## Potential fix
https://github.com/apache/airflow/blob/96697180d79bfc90f6964a8e99f9dd441789177c/airflow/contrib/operators/dataproc_operator.py#L1105
Remove the name postfix from the bucket_name and object_name.
```
GoogleCloudStorageHook(
1103 google_cloud_storage_conn_id=self.gcp_conn_id
1104 ).upload(
1105 bucket=bucket,
1106 object=temp_filename,
1107 mime_type='application/x-python',
1108 filename=local_file
1109 )
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
