Lee-W commented on code in PR #52700:
URL: https://github.com/apache/airflow/pull/52700#discussion_r2196342499
##########
providers/sftp/src/airflow/providers/sftp/hooks/sftp.py:
##########
@@ -86,10 +93,12 @@ def __init__(
self,
ssh_conn_id: str | None = "sftp_default",
host_proxy_cmd: str | None = None,
+ managed_conn: bool = True,
*args,
**kwargs,
) -> None:
self.conn: SFTPClient | None = None
+ self.managed_conn = managed_conn
Review Comment:
I think we probably can check whether `managed_conn` and `conn` are both
falsy here instead of changing the code in every method. Or is there anything I
missed?
##########
providers/sftp/src/airflow/providers/sftp/hooks/sftp.py:
##########
@@ -155,6 +177,18 @@ def get_conn_count(self) -> int:
"""Get the number of open connections."""
return self._conn_count
+ def _describe_directory(self, conn: SFTPClient, path: str) -> dict[str,
dict[str, str | int | None]]:
+ flist = sorted(conn.listdir_attr(path), key=lambda x: x.filename)
+ files = {}
+ for f in flist:
+ modify =
datetime.datetime.fromtimestamp(f.st_mtime).strftime("%Y%m%d%H%M%S") # type:
ignore
+ files[f.filename] = {
+ "size": f.st_size,
+ "type": "dir" if stat.S_ISDIR(f.st_mode) else "file", # type:
ignore
+ "modify": modify,
+ }
+ return files
Review Comment:
```python
class FileMetadata(TypedDict):
file_size: str
file_type: Literal["dir", "file"]
modified_timestamp: str
return {
f.filename: FileMetadata(file_size=..., file_type=...,
modified_timestamp=...)
}
```
I'm surprised the code sample above isn't working. Can you check if it works
for you?
##########
providers/sftp/src/airflow/providers/sftp/hooks/sftp.py:
##########
@@ -34,9 +34,13 @@
import asyncssh
from asgiref.sync import sync_to_async
-from airflow.exceptions import AirflowException,
AirflowProviderDeprecationWarning
+from airflow.exceptions import (
+ AirflowException,
+ AirflowProviderDeprecationWarning,
+)
from airflow.providers.sftp.version_compat import BaseHook
from airflow.providers.ssh.hooks.ssh import SSHHook
+from airflow.providers.standard.exceptions import ConnectionNotOpenedException
Review Comment:
Oh... sorry I should have been more specific. We should add it to `from
airflow.providers.sftp.exceptions` instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
