KostyaEsmukov commented on issue #8151: BugFix: Fix writing Dag Code for Serialized DAGs URL: https://github.com/apache/airflow/pull/8151#issuecomment-609383849 @kaxil Thank you for the patch! I've applied it on my installation, and the `dag_code` table filled with rows. However, new errors appeared in scheduler: ``` Process DagFileProcessor35-Process: Traceback (most recent call last): File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 158, in _run_file_processor pickle_dags) File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1582, in process_file dag.sync_to_db() File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/models/dag.py", line 1519, in sync_to_db DagCode.bulk_sync_to_db([orm_dag.fileloc], session=session) File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 70, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/models/dagcode.py", line 131, in bulk_sync_to_db orm_dag_code.last_updated = timezone.utcnow() UnboundLocalError: local variable 'orm_dag_code' referenced before assignment ``` and ``` Traceback (most recent call last): File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/utils/dag_processing.py", line 624, in _run_processor_manager processor_manager.start() File "/usr/local/lib/python3.7/site-packages/airflow/utils/dag_processing.py", line 856, in start self._refresh_dag_dir() File "/usr/local/lib/python3.7/site-packages/airflow/utils/dag_processing.py", line 928, in _refresh_dag_dir DagCode.remove_deleted_code(self._file_paths) File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/airflow/models/dagcode.py", line 151, in remove_deleted_code cls.fileloc.notin_(alive_dag_filelocs))).delete()) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3789, in delete delete_op.exec_() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 1696, in exec_ self._do_pre_synchronize() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 1778, in _do_pre_synchronize from_=err, File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_ raise exception sqlalchemy.exc.InvalidRequestError: Could not evaluate current criteria in Python: "Cannot evaluate clauselist with operator <function comma_op at 0x7f14acf61440>". Specify 'fetch' or False for the synchronize_session parameter. ``` Do these look like related bugs?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
