joarobles commented on code in PR #29045:
URL: https://github.com/apache/airflow/pull/29045#discussion_r1081620153
##########
airflow/providers/amazon/aws/transfers/sql_to_s3.py:
##########
@@ -134,11 +127,23 @@ def __init__(
raise AirflowException(f"The argument file_format doesn't support
{file_format} value.")
@staticmethod
- def _fix_dtypes(df: pd.DataFrame, file_format: FILE_FORMAT) -> None:
Review Comment:
Done, I've moved only the `DataFrame` import up to the `if TYPE_CHECKING `
since the other imports `Int64Dtype` and `Float64Dtype` are required for type
casting later in the `_fix_dtypes` function
##########
airflow/providers/amazon/aws/transfers/sql_to_s3.py:
##########
@@ -134,11 +127,23 @@ def __init__(
raise AirflowException(f"The argument file_format doesn't support
{file_format} value.")
@staticmethod
- def _fix_dtypes(df: pd.DataFrame, file_format: FILE_FORMAT) -> None:
+ def _fix_dtypes(df, file_format: FILE_FORMAT) -> None:
"""
Mutate DataFrame to set dtypes for float columns containing NaN values.
Set dtype of object to str to allow for downstream transformations.
"""
+ try:
+ import numpy as np
+ from pandas import DataFrame, Float64Dtype, Int64Dtype
+ except ImportError as e:
+ from airflow.exceptions import
AirflowOptionalProviderFeatureException
+
+ raise AirflowOptionalProviderFeatureException(e)
+
+ # DataFrame type checking should be postponed because of imports order
+ if not isinstance(df, DataFrame):
+ raise TypeError('The given dataframe is not a valid pandas
DataFrame')
Review Comment:
Added the type annotation back
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
