fpopic commented on issue #36629:
URL: https://github.com/apache/airflow/issues/36629#issuecomment-1882609788
@RNHTTR @vatsrahul1001 Sorry for closing, but was too early.
My pattern was testing for non-existing file and it worked, but once the
file exists on SFTP server, I am getting
```
[2024-01-09T08:18:29.763+0000] {sftp.py:90} INFO - Found File
XXXX_YYY_KW2351_ZZZZ.WWW last modified: 20231221050445
[2024-01-09 08:18:29,771] {taskinstance.py:1826} ERROR - Task failed with
exception\nTraceback (most recent call last):\n File
"/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py", line 225,
in execute\n raise e\n File
"/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py", line 212,
in execute\n poke_return = self.poke(context)\n File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/sftp/sensors/sftp.py",
line 97, in poke\n _newer_than = convert_to_utc(self.newer_than)\n File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
104, in convert_to_utc\n if not is_localized(value):\n File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
39, in is_localized\n return value.utcoffset() is not None\nAttributeError:
'str' object has no attribute 'utcoffset'
[2024-01-09T08:18:29.771+0000] {taskinstance.py:1826} ERROR - Task failed
with exception
Traceback (most recent call last):
File "/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py",
line 225, in execute
raise e
File "/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py",
line 212, in execute
poke_return = self.poke(context)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/sftp/sensors/sftp.py",
line 97, in poke
_newer_than = convert_to_utc(self.newer_than)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
104, in convert_to_utc
if not is_localized(value):
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
39, in is_localized
return value.utcoffset() is not None
AttributeError: 'str' object has no attribute 'utcoffset'
[2024-01-09 08:18:29,783] {taskinstance.py:1346} INFO - Marking task as
FAILED. dag_id=dag_XXX, task_id=wait_for_sftp_file,
execution_date=20231222T000000, start_date=, end_date=20240109T081829
[2024-01-09T08:18:29.783+0000] {taskinstance.py:1346} INFO - Marking task as
FAILED. dag_id=dag_XXX, task_id=wait_for_sftp_file,
execution_date=20231222T000000, start_date=, end_date=20240109T081829
Traceback (most recent call last):
File "/opt/python3.8/bin/airflow", line 8, in <module>
sys.exit(main())
File "/opt/python3.8/lib/python3.8/site-packages/airflow/__main__.py",
line 48, in main
args.func(args)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/cli/cli_config.py", line
52, in command
return func(*args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/utils/cli.py",
line 112, in wrapper
return f(*args, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 618, in task_test
ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/session.py", line 76,
in wrapper
return func(*args, session=session, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1723, in run
self._run_raw_task(
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/session.py", line 73,
in wrapper
return func(*args, **kwargs)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1408, in _run_raw_task
self._execute_task_with_callbacks(context, test_mode)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1560, in _execute_task_with_callbacks
result = self._execute_task(context, task_orig)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1630, in _execute_task
result = execute_callable(context=context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py",
line 225, in execute
raise e
File "/opt/python3.8/lib/python3.8/site-packages/airflow/sensors/base.py",
line 212, in execute
poke_return = self.poke(context)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/sftp/sensors/sftp.py",
line 97, in poke
_newer_than = convert_to_utc(self.newer_than)
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
104, in convert_to_utc
if not is_localized(value):
File
"/opt/python3.8/lib/python3.8/site-packages/airflow/utils/timezone.py", line
39, in is_localized
return value.utcoffset() is not None
AttributeError: 'str' object has no attribute 'utcoffset'
```
this was with:
```python
wait_for_sftp_file = SFTPSensor(
task_id=f"wait_for_sftp_file",
sftp_conn_id="sftp_musictrace",
path=f"XXXX_YYY_KW{{{{ macros.ds_format(ds, '%Y-%m-%d', '%y%U')
}}}}_ZZZZ.WWW",
newer_than='newer_than='{{ dag_run.logical_date.add(days=-1,
hours=-2) }}',
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
