SameerMesiah97 commented on PR #66220: URL: https://github.com/apache/airflow/pull/66220#issuecomment-4413874978
> > polars has been added as an optional dependency in `pyproject.toml`. A guard has also been added to direct users to install the dependency as well.. > > Just revisiting this now and I noticed that the base `DbApiHook` also has guarded imports for both pandas and polars, but the error messages point users to install via the common-sql package (e.g.,`apache-airflow-providers-common-sql[polars]`). Since the pandas path would fall through to the base hook's exception (because _get_pandas_df isn't overridden here), maybe it actually makes sense to match that same pattern in this override for consistency. That way both df types point users to the same install path. A maintainer would know better but thought it was worth mentioning. Were you able to get `get_pandas_df` to work with the `ElasticsearchSQLHook`? Based on a few sanity checks, it seems that that the Elasticsearch library is not fully DP-API compliant and this makes `get_pandas_df` non-functional as database semantics (for e.g. rollbacks) are not handled by the current implementation. I am actually considering another PR which is essentially doing this what PR is doing but for Pandas instead. That PR would override `_get_pandas_df` and include the same `AirflowOptionalProviderFeatureException` which should make it symmetrical for both pandas and polars. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
