potiuk commented on issue #15010: URL: https://github.com/apache/airflow/issues/15010#issuecomment-865307499
I think it's a deliberate decision to not store the results in Xcom. This is quite a powerful message to the users in fact "DO NOT" use XCom to pass data there. Sometimes this kind of decision is made - to make some actions more difficult rather than easy to discourage the easy (but not correct) path. It's rather simple (but a little involved) to write a custom operator to do something with the result of a query if you want. Then you can read data from DB and do something with it in the same operation. This is what Hooks are for. You can use PostgresHook in your custom operator to read the data, and then use another Hook to write it somewhere else or write your XCom based on results. But you are not encouraged to simply use the operator to run a SELECT * query and pass the result via XCom. Passing results of a select query to XCom, which actually performs another write to the DB makes little sense in Airflow world. Making it too easy makes people fall into bad use patterns. So (it was not me - the decision was made long before I joined Airflow) the decision was made to make it difficult to follow that path. The operator is really intended to run DML or DDL rather than DQL queries. And I think it's better if it stays this way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
