[jira] [Commented] (AIRFLOW-2378) Add Groupon to README
[ https://issues.apache.org/jira/browse/AIRFLOW-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453450#comment-16453450 ] steven casey commented on AIRFLOW-2378: --- https://github.com/apache/incubator-airflow/pull/3267 > Add Groupon to README > - > > Key: AIRFLOW-2378 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2378 > Project: Apache Airflow > Issue Type: Wish >Reporter: steven casey >Assignee: steven casey >Priority: Trivial > > Add Groupon to current list of Airflow users -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2378) Add Groupon to README
[ https://issues.apache.org/jira/browse/AIRFLOW-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] steven casey updated AIRFLOW-2378: -- Description: Add Groupon to current list of Airflow users (was: Add Bonnier Broadcasting to current list of Airflow users) > Add Groupon to README > - > > Key: AIRFLOW-2378 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2378 > Project: Apache Airflow > Issue Type: Wish >Reporter: steven casey >Assignee: steven casey >Priority: Trivial > > Add Groupon to current list of Airflow users -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2378) Add Groupon to README
steven casey created AIRFLOW-2378: - Summary: Add Groupon to README Key: AIRFLOW-2378 URL: https://issues.apache.org/jira/browse/AIRFLOW-2378 Project: Apache Airflow Issue Type: Wish Reporter: steven casey Assignee: Guillermo Rodríguez Cano Add Bonnier Broadcasting to current list of Airflow users -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2378) Add Groupon to README
[ https://issues.apache.org/jira/browse/AIRFLOW-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] steven casey reassigned AIRFLOW-2378: - Assignee: steven casey (was: Guillermo Rodríguez Cano) > Add Groupon to README > - > > Key: AIRFLOW-2378 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2378 > Project: Apache Airflow > Issue Type: Wish >Reporter: steven casey >Assignee: steven casey >Priority: Trivial > > Add Bonnier Broadcasting to current list of Airflow users -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2167) Scheduler's clear_nonexistent_import_errors function should be called on first iteration
Casey created AIRFLOW-2167: -- Summary: Scheduler's clear_nonexistent_import_errors function should be called on first iteration Key: AIRFLOW-2167 URL: https://issues.apache.org/jira/browse/AIRFLOW-2167 Project: Apache Airflow Issue Type: Bug Components: scheduler Affects Versions: 1.9.0 Reporter: Casey Assignee: Casey Attachments: Screen Shot 2018-03-02 at 2.08.29 PM.png In `airflow/jobs.py`, the `**clear_nonexistent_import_errors` function is not called until the amount of seconds defined by `dag_dir_list_interval` has elapsed. If the scheduler is not alive for the duration of `dag_dir_list_interval` (300 seconds) this cleanup never occurs. In some environments this could result in error messages displaying on the UI permanently, even if the DAG has been removed from the environment. It was previously an Airflow best practice to have the scheduler run N runtimes and terminate. Then, the scheduler would started again by an auxiliary process like Docker or Supervisor. This situation is what brought the bug to my attention. My suggested fix is to tweak jobs.py to run the import error cleanup on the first iteration and periodically as defined by `dag_dir_list_interval`. This way, a scheduler setup with a small number of runs will still have old errors cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)