dimon222 edited a comment on issue #17897:
URL: https://github.com/apache/airflow/issues/17897#issuecomment-910671083
The issue was initially noticed on 2.1.2 version, but I have given it a test
on version 2.1.3 now and I'm having it.
Perhaps, this might point in this direction:
1. Store dags in DB option is enabled in airflow.cfg
2. To make scheduler crash it should be a lot of dags with this problem so
that all processes that are doing parsing kicked in simultaneously and raised
exception. I tried it with just single dag and it didn't crash the scheduler
(tho, spammed a lot of stacktraces in scheduler and the dag is essentially
broken).
The exact stacktrace is very long but here's the most important part of it:
```
...
File "/venv/lib/python3.6/site-packages/airflow/models/dag.py", line 1920,
in bulk_write_to_db
DagCode.bulk_sync_to_db([dag.fileloc for dag in orm_dags])
...
File "/venv/lib/python3.6/site-packages/MySQLdb/connections.py", line 259,
in query
_mysql.connection.query(self, query)
sqlalchemy.exc.IntegrityError: (MySQLdb._exceptions.IntegrityError) (1062,
"Duplicate entry "Test-testdag" for key 'PRIMARY'")
[SQL: INSERT INTO dag_tag (name, dag_id) VALUES (%s, %s)]
[parameters: ('Test', 'testdag')]
(Background on this error at: http://sqlalche.me/e/13/gpkj )
```
The dag initially had tags `['test']` and then the tag listing was changed
to `['Test']`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
