john-jac opened a new pull request #19137: URL: https://github.com/apache/airflow/pull/19137
Airflow users wishing to trigger Amazon Redshift queries today have to use a Postgres connection. While effective, this has the limitation that the Redshift cluster must expose a Postgres endpoint that is accessible to the Airflow environment. As an alternative, the AWS Boto3 library exposes a client called [redshift-data](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/redshift-data.html) that allows users to query and retrieve data via the AWS API and avoid the necessity of a Postgres connection. This PR adds a RedshiftDataHook to the existing Redshift provider, along with tests and an example, which wraps the "redshift-data" Boto3 client and allows users to run Redshift queries without a Postgres connection. In the future, this capability could be added to any existing Redshift transfer operator, with an option to auto-detect whether to use Postgres or Boto3 based upon the connection type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
