Hi everyone,
We have been using airflow for roughly about 7 months. Pretty smooth :)
Thanks!
Today when I build my airflow container, I found a weird issue.
This is install command for airflow.
ENV AIRFLOW_VERSION 1.7.0RUN sudo pip install airflow==${AIRFLOW_VERSION}
Now the problem happens with 'sqlalchemy '.
While installing airflow, I saw this in the logs.
*Collecting sqlalchemy>=0.9.8 (from airflow==1.7.0)*
* Downloading SQLAlchemy-1.1.2.tar.gz (5.1MB)*
When I start the scheduler, I see a lot of errors (below stack trace).
This is happening when I am creating pools (programmatically).
I think we are hitting this issue:
http://docs.sqlalchemy.org/en/latest/changelog/migration_11.html
(sqlalchemy has a new release on Oct 17th)
I am not sure if this is a bug.
Looking at airflow/setup.py,
I find install_requires has "'sqlalchemy>=0.9.8'"
Shouldn't this be "'sqlalchemy>=0.9.8, <=1.1'" ?
If not, has somebody faced a similar issue? Any pointers?
*Stack trace: *
When I started the scheduler process, this is the error I got:
[2016-10-19 07:21:56,073] {jobs.py:642} ERROR - Object blah_pool is not
legal as a SQL literal value
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 639,
in _execute
self.process_dag(dag, executor)
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 488,
in process_dag
elif ti.is_runnable(flag_upstream_failed=True):
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line
770, in is_runnable
return self.is_queueable(flag_upstream_failed) and not self.pool_full()
File "/usr/local/lib/python2.7/dist-packages/airflow/utils.py", line 142,
in wrapper
result = func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line
939, in pool_full
.filter(Pool.pool == self.task.pool)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/operators.py", line
304, in __eq__
return self.operate(eq, other)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line
175, in operate
return op(self.comparator, *other, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/operators.py", line
304, in __eq__
return self.operate(eq, other)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/properties.py", line
270, in operate
return op(self.__clause_element__(), *other, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/annotation.py", line
95, in __eq__
return self.__element.__class__.__eq__(self, other)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/operators.py", line
304, in __eq__
return self.operate(eq, other)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py",
line 686, in operate
return op(self.comparator, *other, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/operators.py", line
304, in __eq__
return self.operate(eq, other)
File "<string>", line 1, in <lambda>
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/type_api.py",
line 63, in operate
return o[0](self.expr, op, *(other + o[1:]), **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/default_comparator.py",
line 64, in _boolean_compare
obj = _check_literal(expr, op, obj)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/default_comparator.py",
line 305, in _check_literal
return expr._bind_param(operator, other, type_=bindparam_type)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py",
line 3823, in _bind_param
unique=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py",
line 1094, in __init__
_compared_to_operator, value)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/type_api.py",
line 516, in coerce_compared_value
_coerced_type = _resolve_value_to_type(value)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/sqltypes.py",
line 2467, in _resolve_value_to_type
"Object %r is not legal as a SQL literal value" % value)
ArgumentError: Object monitor_pool is not legal as a SQL literal value
