Dev-iL commented on code in PR #74817:
URL: https://github.com/apache/airflow/pull/74817#discussion_r4307062449
##########
providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:
##########
@@ -220,7 +221,23 @@ def _strip_suffixes(
return path[: -len(suffix)]
return path
+ def _ensure_parquet_engine(self) -> None:
+ needs_parquet = self.file_format == FILE_FORMAT.PARQUET or (
+ ("dtype_backend", "pyarrow") in self.read_kwargs.items()
+ )
+ if self.df_type != "pandas" or not needs_parquet:
+ return
+ if any(importlib.util.find_spec(engine) for engine in ("pyarrow",
"fastparquet")):
+ return
+
+ from airflow.providers.common.compat.sdk import
AirflowOptionalProviderFeatureException
Review Comment:
Done — folded into the existing top-level `common.compat.sdk` import.
Checking that none was left turned up a second one: `_fix_dtypes` has
carried its own local import of the same exception since 4d3c5ead3b5, and the
module-level import this PR adds makes it dead. Removed that as well, since
your reasoning applies to it unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
