potiuk commented on PR #32319: URL: https://github.com/apache/airflow/pull/32319#issuecomment-1826875811
I do not think it has anything to do with "standard" behaviour. This is an internal implementation of Hooks - each DB hooks might make their own decision on how to make the (already common - via common.sql) to make the rows returned as serializable variant. Could you please elaborate what do you mean by "missing versioning" and how you want to implement what it does? It is nothing that Airflow's serialization should be concerned about - this is really "standardising" behaviour of the DBAPI implementation - Python's DBAPI does not have very "strong" guarantees about what is returned, and some of the implementation (like Databricks) chose to implement to return non-serializable objects while most of the other DBAPI implememtations chose to use Tuples of rows + Tuples of descriptions which are serializable. The big proble we were trying to solve with Common.sql is to introduce common interface of what alll DBHooks will be returning. In this case it's really not even something that IMHO "airflow" serialization should be concerned about (if this is what you are after). This is purely one-way serializing - we just want to make sure that whatever gets returned via DBAPI calls (and essentially via Hook) is: a) standard according to what our DBHook should return (i.e. following the tuple structure + tuples of metadata describing the rows b) is serializable In this sense - we do not need anything else - like versioning - we just need to find a way to make what gets returned can be returned via XCom. But maybe I misunderstand what you want to achieve @bolkedebruin ? Could you please show an example of what you want to get and how what we want to achieve by that one is done? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
