[
https://issues.apache.org/jira/browse/AIRFLOW-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551988#comment-16551988
]
ASF subversion and git services commented on AIRFLOW-2778:
----------------------------------------------------------
Commit ba5b67b6c99e0bfb0f827518cdb31b5eb5a5a626 in incubator-airflow's branch
refs/heads/master from [~knyuen]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=ba5b67b ]
[AIRFLOW-2778] Explicit import for dag_processing.list_py_file_paths
The use of utils.dag_processing.list_py_file_paths causes a failure if
utils.dag_processing is not already loaded indirectly.
> Bad Import in collect_dag in DagBag
> -----------------------------------
>
> Key: AIRFLOW-2778
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2778
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10
> Reporter: Kevin Yuen
> Assignee: Kevin Yuen
> Priority: Minor
>
> I run the following test to make sure there are no import errors in our CI.
> After upgrading to 1.10-stable from 1.9.0, the following test starting
> failing:
> {code:java}
> class TestAirflowDag:
> def test_dagbag_import(self):
> """Verify that Airflow will be able to import all DAGS in the repo
> """
> dagbag = self._get_dagbag()
> assert len(dagbag.import_errors) == 0
> def _get_dagbag(self):
> dag_folder = os.getenv('AIRFLOW_DAGS', False)
> assert dag_folder is not None
> return DagBag(dag_folder=dag_folder, include_examples=False)
> {code}
> The following error was raised:
> {code:java}
> # Used to store stats around DagBag processing
> stats = []
> FileLoadStat = namedtuple(
> 'FileLoadStat',
> "file duration dag_num task_num dags"
> )
> for filepath in utils.dag_processing.list_py_file_paths(dag_folder):
> E AttributeError: module 'airflow.utils' has no attribute 'dag_processing'
> {code}
> The issue is likely because `dag_processing` was not imported.
> [https://github.com/apache/incubator-airflow/blob/7f2bc0ddf74c9bf9113401c4ecb1355e6c2fab7f/airflow/models.py#L68-L94]
> This was further validated by stopping execution using pdb, and running
> before utils.dag_processing.list_py_file_paths(dag_folder) is called.
> {code:java}
> >> from airflow.utils.dag_processing import *{code}
> The test then proceed to pass since the file was loaded.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
