[
https://issues.apache.org/jira/browse/AIRFLOW-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Felix Uellendall resolved AIRFLOW-4588.
---------------------------------------
Resolution: Fixed
> Add GoogleDiscoveryApiHook and GoogleApiToS3Transfer
> ----------------------------------------------------
>
> Key: AIRFLOW-4588
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4588
> Project: Apache Airflow
> Issue Type: New Feature
> Components: contrib, gcp
> Affects Versions: 1.10.3
> Reporter: Felix Uellendall
> Assignee: Felix Uellendall
> Priority: Major
> Fix For: 1.10.6
>
>
> This PR adds a hook based on Google's Discovery API to communicate to any
> Google Services.
> And also a implementation of this hook in an operator that transfers data
> from Google to S3.
> - add documentation to integration.rst
> The hook provides:
> - a get_conn function to authenticate to the Google API via an airflow
> connection
> - a query function to dynamically query all data available for a specific
> endpoint and given parameters. (You are able to either retrieve one page of
> data or all data)
> The transfer operator provides:
> - basic transfer between google api and s3
> - passing an xcom variable to dynamically set the endpoint params for a
> request
> - exposing the response data to xcom, but raises exception when it exceeds
> MAX_XCOM_SIZE
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
