hussein-awala commented on issue #31753: URL: https://github.com/apache/airflow/issues/31753#issuecomment-1581309762
> What's the point? Using the hook as in the original post is way shorter and easier. Edit: or you mean just for the test? Here is the full comment: > I believe it's a specific issue with the Databricks provider, where the Row class does not allow access to private attributes and methods. You can see the relevant code [here](https://github.com/databricks/databricks-sql-python/blob/6f8314459736762cabab59ddfaf743dc41030c96/src/databricks/sql/types.py#LL159C1-L170C39). However, the new serializer introduced in version 2.6.0 (refactored in https://github.com/apache/airflow/pull/28067) requires access to the method name in order to serialize the object. Can you try to extend the operator class, and override the run method to convert the output from a list of Row to a list of tuples or dicts? If we cannot serialize the returned objects from the Databricks cursor because of a limitation in the databricks lib, we need to find a workaround for Databricks provider and not change our serialization strategy, for that I suggest this solution before opening a PR. So it is not just for testing, if you test it and you find that it fixes the issue, you can open a PR to fix it in the Airflow Databricks provider. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
