potiuk commented on pull request #17514:
URL: https://github.com/apache/airflow/pull/17514#issuecomment-895472650


   BTW. The reason I think this might be the problem is this: 
   
   Failed job: 
https://github.com/apache/airflow/runs/3283082832?check_suite_focus=true
   
   This is one of the very few failed jobs of this kind that - even if there 
are no logs available in the UI, we still can download raw logs and you can see 
what happened:
   
   
https://pipelines.actions.githubusercontent.com/dG18C1r9Veh0Ng8eypyIHvAXk1Mz6Bvz6mj1myj6BZplhvMhGo/_apis/pipelines/1/runs/90761/signedlogcontent/66?urlExpires=2021-08-09T19%3A07%3A35.8890404Z&urlSigningMethod=HMACV1&urlSignature=EM7eqBqs1iMQEzLXTJuM7MWU8NOV1%2BifLNSVeCj4WjU%3D
   
   
   In this case, this happened:
   
   ```
   2021-08-09T18:20:47.6858820Z Test type: WWW succeeded.
   2021-08-09T18:20:47.6859157Z 
   2021-08-09T18:20:47.6859370Z 
   2021-08-09T18:20:47.6859897Z ##### The WWW succeeded ##### 
   2021-08-09T18:20:47.6860200Z 
   2021-08-09T18:20:47.6860606Z 
   2021-08-09T18:20:47.6861549Z ##[endgroup]
   2021-08-09T18:22:24.0406973Z ##[error]The operation was canceled.
   2021-08-09T18:23:03.9854824Z Post job cleanup.
   2021-08-09T18:23:33.3720207Z ##[error]The runner has received a shutdown 
signal. This can happen when the runner service is stopped, or a manually 
started runner is canceled.
   2021-08-09T18:23:33.4888643Z Cleaning up orphan processes
   ```
   
   All the parallel test runs completed successfully. from the logs seems that 
all of them completed, but then the job was canceled some time after that. The 
only reason I can see this happens, is that the semaphores for the parallel run 
were somehow stale. I cleaned them up before, but this cleanup was "gentle" - I 
added force cleaning and remove EVERYTHING after some gentle cleaning and 
killing all processes potentially stale from previous run.. This might end up 
(still) with some processes that could not be killed but at least they will not 
block parallel from completing the parallel run.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to