Taragolis commented on code in PR #29079:
URL: https://github.com/apache/airflow/pull/29079#discussion_r1083313280
##########
airflow/providers/common/sql/hooks/sql.py:
##########
@@ -197,8 +197,7 @@ def get_pandas_df(self, sql, parameters=None, **kwargs):
"'apache-airflow-providers-common-sql[pandas]'."
)
- with closing(self.get_conn()) as conn:
- return psql.read_sql(sql, con=conn, params=parameters, **kwargs)
+ return psql.read_sql(sql, con=self.get_sqlalchemy_engine(),
params=parameters, **kwargs)
Review Comment:
Better explicit open connection, and close it. I don't think pandas actually
do this stuff.
As well as dispose SQLAlchemy Engine
```suggestion
engine = self.get_sqlalchemy_engine()
try:
with engine.connect() as conn:
return psql.read_sql(sql, con=conn, params=parameters,
**kwargs)
finally:
engine.dispose()
```
##########
airflow/providers/common/sql/hooks/sql.py:
##########
@@ -218,8 +217,9 @@ def get_pandas_df_by_chunks(self, sql, parameters=None, *,
chunksize, **kwargs):
"'apache-airflow-providers-common-sql[pandas]'."
)
- with closing(self.get_conn()) as conn:
- yield from psql.read_sql(sql, con=conn, params=parameters,
chunksize=chunksize, **kwargs)
+ yield from psql.read_sql(
+ sql, con=self.get_sqlalchemy_engine(), params=parameters,
chunksize=chunksize, **kwargs
+ )
Review Comment:
The same here: create connection -> close it (context manger) -> yielding
chunks -> dispose engine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
