KulykDmytro opened a new issue, #29737:
URL: https://github.com/apache/airflow/issues/29737

   ### Apache Airflow version
   
   2.5.1
   
   ### What happened
   
   Following metric `dag_processing.total_parse_time` exposing smth not 
relevant to overall processing time
   
   
![image](https://user-images.githubusercontent.com/34435869/221139873-03f9623f-76eb-40bb-98f8-10e5950c1cd5.png)
   ```
   DAG File Processing Stats                                                    
                                                 
                                                                                
                                                 
   File Path                                            PID  Runtime      # 
DAGs    # Errors  Last Runtime    Last Run           
   -------------------------------------------------  -----  ---------  
--------  ----------  --------------  -------------------
   /opt/airflow/dags/player_time_features.py                                   
1           0  0.89s           2023-02-24T09:18:59
   /opt/airflow/dags/maxmind_uploader.py                                       
1           0  0.81s           2023-02-24T09:19:29
   /opt/airflow/dags/player_team_similarities.py                               
1           0  1.34s           2023-02-24T09:18:31
   /opt/airflow/dags/aws_sanitize.py                                           
1           0  3.58s           2023-02-24T09:18:58
   /opt/airflow/dags/ad_backoffice.py                                          
1           0  1.05s           2023-02-24T09:19:28
   /opt/airflow/dags/gdpr.py                                                   
1           0  0.46s           2023-02-24T09:18:39
   /opt/airflow/dags/aws_regional_split_backfill.py                            
1           0  2.55s           2023-02-24T09:19:12
   /opt/airflow/dags/reconciliation_daily.py                                   
1           0  1.04s           2023-02-24T09:19:16
   /opt/airflow/dags/maintenance.py                                            
1           0  0.24s           2023-02-24T09:17:51
   /opt/airflow/dags/athena_repair_partitions.py                               
1           0  0.81s           2023-02-24T09:19:28
   /opt/airflow/dags/trending_events.py                                        
1           0  1.74s           2023-02-24T09:19:07
   /opt/airflow/dags/casino_games_similarities.py                              
1           0  1.17s           2023-02-24T09:18:18
   /opt/airflow/dags/player_segmentation.py                                    
1           0  1.44s           2023-02-24T09:17:41
   /opt/airflow/dags/test_copy.py                                              
1           0  0.71s           2023-02-24T09:19:01
   /opt/airflow/dags/datahub_ingest.py                                         
1           0  0.90s           2023-02-24T09:19:05
   /opt/airflow/dags/gs_exporter.py                                            
1           0  2.65s           2023-02-24T09:17:37
   /opt/airflow/dags/canary.py                                                 
1           0  1.75s           2023-02-24T09:19:04
   /opt/airflow/dags/sfsc_load.py                                              
1           0  1.06s           2023-02-24T09:19:15
   /opt/airflow/dags/led_recognition_processed.py                              
1           0  2.28s           2023-02-24T09:19:09
   /opt/airflow/dags/casino_games_recommendations.py                           
1           0  1.77s           2023-02-24T09:18:59
   /opt/airflow/dags/vip_export.py                                             
1           0  1.29s           2023-02-24T09:18:29
   /opt/airflow/dags/autotest.py                                               
2           0  0.63s           2023-02-24T09:19:30
   /opt/airflow/dags/anomaly_detection.py                                      
2           0  3.67s           2023-02-24T09:17:37
   /opt/airflow/dags/native_rls.py                                             
3           0  1.14s           2023-02-24T09:19:26
   /opt/airflow/dags/dbt.py                                                    
4           0  1.23s           2023-02-24T09:18:14
   /opt/airflow/dags/front_data.py                                             
4           0  8.55s           2023-02-24T09:18:10
   /opt/airflow/dags/gs_loader.py                                              
4           0  1.37s           2023-02-24T09:19:25
   /opt/airflow/dags/backfill_lambda.py                                        
6           0  1.91s           2023-02-24T09:19:14
   /opt/airflow/dags/tableau_refresher.py                                      
7           0  1.85s           2023-02-24T09:17:39
   /opt/airflow/dags/extract_load.py                                           
8           0  1.30s           2023-02-24T09:18:49
   /opt/airflow/dags/validator.py                                             
30           0  1.91s           2023-02-24T09:19:24
   /opt/airflow/dags/aws_transforms.py                 1768  34.15s           
56           0  151.91s         2023-02-24T09:17:01
   ```
   
   
   
   ### What you think should happen instead
   
   _No response_
   
   ### How to reproduce
   
   process dags by dag-processor
   observe exposed metric: dag_processing.total_parse_time
   
   ### Operating System
   
   n/a
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   AWS EKS
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to