ephraimbuddy commented on PR #48004: URL: https://github.com/apache/airflow/pull/48004#issuecomment-2745112088
> https://github.com/giampaolo/psutil/blob/a17550784b0d3175da01cdb02cee1bc6b61637dc/psutil/__init__.py#L766-L773 > > If you are worried about process reuse and the start_time being wrong, then call `proc.create_time()` once first. > > However the chance of process reuse happening and being the cause of the bugs is tiny. I'm not saying it can't happen, but that the for it to, a large number of processes need to have been created, enough for the pid to overflow and wrap around. I don't know what the limit is, but I suspect it's somewhere around 64k processes on OSX. I was able to determine that it’s not process reuse by printing out the previous PIDs before process creation and checking it with the new ID. My observation is that any process created after the heartbeat recovery message of say recovered after 290ms use the time before the recovery instead of the time after the recovery which is when the process started being used for file processing. So instead of using the process start time, we can use the time that the process started processing the files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
