Hi,
I am new to Airflow. I set up a local Airflow instance with CeleryExecutor.
When I execute the command:
airflow backfill remote_execution_test -s 2014-07-01 -e 2014-07-02
I get the following error in the worker logs:
[2016-06-16 17:03:48,753: ERROR/Worker-9] Command 'airflow run
remote_execution_test print_date 2014-07-01T00:00:00 --pickle 7 --local '
returned non-zero exit status 1
with the following stack trace:
Traceback (most recent call last):
File "/Users/vineet/.virtualenvs/airflow/bin/airflow", line 15, in
<module>
args.func(args)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/airflow/bin/cli.py",
line 211, in run
DagPickle).filter(DagPickle.id == args.pickle).first()
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2659, in first
ret = list(self[0:1])
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2457, in __getitem__
return list(res)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/loading.py",
line 86, in instances
util.raise_from_cause(err)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/loading.py",
line 71, in instances
rows = [proc(row) for row in fetch]
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/loading.py",
line 428, in _instance
loaded_instance, populate_existing, populators)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/orm/loading.py",
line 486, in _populate_full
dict_[key] = getter(row)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py",
line 1253, in process
return loads(value)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/dill/dill.py",
line 260, in loads
return load(file)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/dill/dill.py",
line 250, in load
obj = pik.load()
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 858, in load
dispatch[key](self)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1090, in load_global
klass = self.find_class(module, name)
File
"/Users/vineet/.virtualenvs/airflow/lib/python2.7/site-packages/dill/dill.py",
line 406, in find_class
return StockUnpickler.find_class(self, module, name)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1124, in find_class
__import__(module)
ImportError: No module named unusual_prefix_remote_execution_test
I am trying to understand what I might be doing wrong here. I tried to run
the command that celery is trying to run without the --pickle flag and it
works.
Note: I am running a single worker on the same machine.
Thanks!
