[
https://issues.apache.org/jira/browse/AIRFLOW-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936594#comment-16936594
]
ASF GitHub Bot commented on AIRFLOW-774:
----------------------------------------
ashb commented on pull request #6157: [AIRFLOW-774] Fix long-broken DAG parsing
Statsd metrics
URL: https://github.com/apache/airflow/pull/6157
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> dagbag_size/collect_dags/dagbag_import_errors stats incorrect
> -------------------------------------------------------------
>
> Key: AIRFLOW-774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-774
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
> Reporter: Dan Davydov
> Assignee: Ash Berlin-Taylor
> Priority: Major
>
> After the multiprocessor change was made (dag folders are processed in
> parallel), the number of dags reported by airflow is for each of these
> subprocesses which is inaccurate, and potentially orders of magnitude less
> than the actual number of dags. These individual processes stats should be
> aggregated. The collect_dags/dagbag_import_errors stats should also be fixed
> (time it takes to parse the dags).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
