[
https://issues.apache.org/jira/browse/AIRFLOW-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103572#comment-17103572
]
ASF GitHub Bot commented on AIRFLOW-2310:
-----------------------------------------
abdulbasitds edited a comment on pull request #6007:
URL: https://github.com/apache/airflow/pull/6007#issuecomment-626255328
> > Appearenlty, some unittest are failing. I did a pytest in morning and
reworked, every files was succeeding all test. havent changed anything but now
in base aws class there is an error.
> > It is now failing on my local breeze as well, Trying to see if i can
find the reason, but cant find any change that I have made
>
> Your PR was not in sync with the master so most likely a related changed
might have been merged too. So make sure to rebase to master frequently.
Yes I just checked that after rebasing, there is problem with an commit that
was made to base_aws on 4rth April by @baolsen involving following lines
```
if not (self.client_type or self.resource_type):
raise AirflowException(
'Either client_type or resource_type'
' must be provided.')
```
In base aws hook client_type and resource_type are optional, like
```
client_type: Optional[str] = None,
resource_type: Optional[str] = None,
```
but it raises exception if not provided. May be i need to pass client_type
somehow. But as per my (vague) understanding, if they are not required, this if
check should not be there
Okay, I needed to add client_type='glue' in hook to super.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Enable AWS Glue Job Integration
> -------------------------------
>
> Key: AIRFLOW-2310
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2310
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Reporter: Olalekan Elesin
> Assignee: Olalekan Elesin
> Priority: Major
> Labels: AWS
>
> Would it be possible to integrate AWS Glue into Airflow, such that Glue jobs
> and ETL pipelines can be orchestrated with Airflow
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
