dinigo opened a new pull request #17592: URL: https://github.com/apache/airflow/pull/17592
closes: #13750 related: #17315 `SqlSensor` relies on underlying hooks that are automatically injected (DI) depending on the connection type provided to the `SqlSensor`. However, from time to time (but mainly in `BigqueryHook`) it is needed to pre-configure the "to-be-injected" hook because some default params do not fit the current config. For example, if using SqlSensor with Bigquery it by default configures the SQL dialect to "legacy SQL", and there is no way to change this through configuration. Instead the SqlSensor has to be extended and some functions overwritten (as mentioned in the #13750). To be able to customise the underlying hook there are 2 approaches to fix this: * Bring your own hook: instantiate a hook and provide it to the sensor already configured so it just checks that it inherits from `DbApiHook` but doesn't try to instantiate on its own. * Pass over the params you want SqlSensor to use to instantiate the hook. Either one has it's drawbacks: * Cannot be used if DAG is defined with yaml/json/config-file, since to instantiate a Hook requires "intelligence", relying solely in config seems more maintainable in the future. * The "tell us what's wrong, we got you covered" does not follow very well the "[open but closed](https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle)" principle. Knowing that, and that @uranusjr approves this second approach, and also given the known limitations, this PR makes the SqlSensor pass over the hook_kwargs to the connection hook locator that returns an instantiated hook. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
