t oo created AIRFLOW-6437:
-----------------------------
Summary: sql filters - remove in (NULL)
Key: AIRFLOW-6437
URL: https://issues.apache.org/jira/browse/AIRFLOW-6437
Project: Apache Airflow
Issue Type: Improvement
Components: core
Affects Versions: 1.10.7
Reporter: t oo
looking at sql generated by sqlalchemy, example query:
2020-01-03 09:10:25,373 INFO sqlalchemy.engine.base.Engine SELECT
task_instance.try_number AS task_instance_try_number, task_instance.task_id AS
task_instance_task_id, task_instance.dag_id AS task_instance_dag_id,
task_instance.execution_date AS task_instance_execution_date,
task_instance.start_date AS task_instance_start_date, task_instance.end_date AS
task_instance_end_date, task_instance.duration AS task_instance_duration,
task_instance.state AS task_instance_state, task_instance.max_tries AS
task_instance_max_tries, task_instance.hostname AS task_instance_hostname,
task_instance.unixname AS task_instance_unixname, task_instance.job_id AS
task_instance_job_id, task_instance.pool AS task_instance_pool,
task_instance.queue AS task_instance_queue, task_instance.priority_weight AS
task_instance_priority_weight, task_instance.operator AS
task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm,
task_instance.pid AS task_instance_pid, task_instance.executor_config AS
task_instance_executor_config
FROM task_instance
WHERE task_instance.dag_id = ? AND task_instance.execution_date = ? AND
(task_instance.state IN (?, *NULL*, ?) OR task_instance.state IS NULL)
2020-01-03 09:10:25,374 INFO sqlalchemy.engine.base.Engine
('example_bash_operator', '2020-01-01 00:00:00.000000', 'up_for_reschedule',
'up_for_retry')
the bolded part should not go into the query
fix is to change below parts to rewrite the states list to not have null before
passing to the query
grep 'if None in' -R *
airflow/models/dagrun.py: if None in state:
airflow/models/dag.py: if None in state:
airflow/models/dag.py: if None in states:
airflow/jobs/scheduler_job.py: if None in states:
airflow/jobs/scheduler_job.py: if None in acceptable_states:
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
