zoid-w commented on issue #31743: URL: https://github.com/apache/airflow/issues/31743#issuecomment-1602197702
Can confirm that the caching mechanism is indeed the reason for this behavior. if you would like to bypass it you could mitigate this in a practical sense by renaming the file as well since the path to the class would not be cached then (possibly some people might not have cli access to the triggerer service or something). On a seperate note : Looking at the [lines](https://github.com/apache/airflow/blob/1d564adc1c5dc31d0c9717d608250b60f9742acb/airflow/jobs/triggerer_job_runner.py#L711-L713) mentioned by @Lee-W, will the dictionary for the cached paths not grow indefinitely? As far as I can see there is no timed and/or triggered cleanup on that part. I assume this will never really get to much out of proportion, but since the triggerer is intended to run 1000 triggers, maybe it is something to consider. My knowledge of async coding in python is quite limited, but maybe a time-based release of cached paths might be a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
