justinpakzad commented on PR #66220: URL: https://github.com/apache/airflow/pull/66220#issuecomment-4413900003
> Were you able to get `get_pandas_df` to work with the `ElasticsearchSQLHook`? Based on a few sanity checks, it seems that that the Elasticsearch library is not fully DP-API compliant and this makes `get_pandas_df` non-functional as database semantics (for e.g. rollbacks) are not handled by the current implementation. I am actually considering another PR which is essentially doing this what PR is doing but for Pandas instead. That PR would override `_get_pandas_df` and include the same `AirflowOptionalProviderFeatureException` which should make it symmetrical for both pandas and polars. Good shout. I didn't actually test the `get_pandas_df` with the Elastic hook, I was just going based on what I had initially observed. The additional PR make sense and that would address the misalignment between the optional provider messages (as you mentioned). Nice one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
