[ https://issues.apache.org/jira/browse/AIRFLOW-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387346#comment-16387346 ]
Damian Momot commented on AIRFLOW-2175: --------------------------------------- This DAG is no longer in dag files but still exists in DB. For 99% it wasn't sub dag. It was probably left in such strange state. As I remember this installation was initialized on Airflow 1.8.1, then upgraded to 1.8.2. Field is nullable in DB, DB is MySQL {code:java} DESCRIBE dag; Field Type Null Key Extra ... 'fileloc' 'varchar(2000)' 'YES' '' NULL '' {code} As a workaround I'll just manually alter record in DB, but null check is probably a good idea, especially that field is nullable in DB - it might affect others > Failed to upgradedb 1.8.2 -> 1.9.0 > ---------------------------------- > > Key: AIRFLOW-2175 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2175 > Project: Apache Airflow > Issue Type: Bug > Components: db > Affects Versions: 1.9.0 > Reporter: Damian Momot > Priority: Critical > > We've got airflow installation with hundreds of DAGs and thousands of tasks. > During upgrade (1.8.2 -> 1.9.0) we've got following error. > After analyzing stacktrace i've found that it's most likely caused by None > value in 'fileloc' field of Dag column. I checked database and indeed we've > got one record with such value: > > > {code:java} > SELECT COUNT(*) FROM dag WHERE fileloc IS NULL; > 1 > SELECT COUNT(*) FROM dag; > 343 > {code} > > > {code:java} > Traceback (most recent call last): > File "/usr/local/bin/airflow", line 27, in <module> > args.func(args) > File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 913, > in upgradedb > db_utils.upgradedb() > File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 320, > in upgradedb > command.upgrade(config, 'heads') > File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 174, > in upgrade > script.run_env() > File "/usr/local/lib/python2.7/dist-packages/alembic/script/base.py", line > 416, in run_env > util.load_python_file(self.dir, 'env.py') > File "/usr/local/lib/python2.7/dist-packages/alembic/util/pyfiles.py", line > 93, in load_python_file > module = load_module_py(module_id, path) > File "/usr/local/lib/python2.7/dist-packages/alembic/util/compat.py", line > 79, in load_module_py > mod = imp.load_source(module_id, path, fp) > File "/usr/local/lib/python2.7/dist-packages/airflow/migrations/env.py", > line 86, in <module> > run_migrations_online() > File "/usr/local/lib/python2.7/dist-packages/airflow/migrations/env.py", > line 81, in run_migrations_online > context.run_migrations() > File "<string>", line 8, in run_migrations > File > "/usr/local/lib/python2.7/dist-packages/alembic/runtime/environment.py", line > 807, in run_migrations > self.get_context().run_migrations(**kw) > File "/usr/local/lib/python2.7/dist-packages/alembic/runtime/migration.py", > line 321, in run_migrations > step.migration_fn(**kw) > File > "/usr/local/lib/python2.7/dist-packages/airflow/migrations/versions/cc1e65623dc7_add_max_tries_column_to_task_instance.py", > line 63, in upgrade > dag = dagbag.get_dag(ti.dag_id) > File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 232, > in get_dag > filepath=orm_dag.fileloc, only_if_updated=False) > File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 249, > in process_file > if not os.path.isfile(filepath): > File "/usr/lib/python2.7/genericpath.py", line 29, in isfile > st = os.stat(path) > TypeError: coercing to Unicode: need string or buffer, NoneType found{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)