So I upgraded my airflow instance to 1.8.0 and found that several of my
DAGs wouldn't load due to this exception:
[2017-03-29 07:04:25,294] [5] {models.py:266} ERROR - Failed to import:
/opt/airflow/dags/automator-sensor.py
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/airflow/models.py", line
263, in process_file
m = imp.load_source(mod_name, filepath)
File "/usr/lib/python3.4/imp.py", line 171, in load_source
module = methods.load()
File "<frozen importlib._bootstrap>", line 1220, in load
File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
File "<frozen importlib._bootstrap>", line 1129, in _exec
File "<frozen importlib._bootstrap>", line 1471, in exec_module
File "<frozen importlib._bootstrap>", line 321, in
_call_with_frames_removed
File "/opt/airflow/dags/automator-sensor.py", line 11, in <module>
from lib.data_loading import DataLoader
File "/opt/airflow/dags/lib/data_loading.py", line 16, in <module>
from airflow.operators.sensors import S3KeySensor
File
"/usr/local/lib/python3.4/dist-packages/airflow/operators/sensors.py", line
33, in <module>
from airflow.hooks.hdfs_hook import HDFSHook
File "/usr/local/lib/python3.4/dist-packages/airflow/hooks/hdfs_hook.py",
line 20, in <module>
from snakebite.client import Client, HAClient, Namenode,
AutoConfigClient
File "/usr/local/lib/python3.4/dist-packages/snakebite/client.py", line
1473
baseTime = min(time * (1L << retries), cap);
^
SyntaxError: invalid syntax
I had to actually uninstall snakebite to get my DAGs to load properly. Has
anyone else encountered this and is there a workaround other than
uninstalling the library? I didn't find an issue on JIRA for it either,
happy to submit a ticket if I'm not missing anything.
-N
[email protected]
--
Read our founder's story.
<https://collectivehealth.com/blog/started-collective-health/>
*This message may contain confidential, proprietary, or protected
information. If you are not the intended recipient, you may not review,
copy, or distribute this message. If you received this message in error,
please notify the sender by reply email and delete this message.*
