potiuk commented on PR #28711: URL: https://github.com/apache/airflow/pull/28711#issuecomment-1435670553
@hussein-awala - I am afraid @ashb is right. DagFileProcessor executes more stuff. It performs sla management, also when DagFileProcessorProcess gets started it receives callbacks to execute for that particular file. Those are the callbacks that are generated elsewhere (allways when callbacks cannot be executed within the process that generates the callback for whatever reason) and stored in the database. Then whenever DagFileProcessor processes a given file, it will execute the callbacks. So the idea that you can stop DagFileProcessor and run it "when needed" is not good as @ashb mentioned. Unless we introduce some other entity that can process those callbacks of course. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
