GuillaumeDubois1 opened a new issue #11329:
URL: https://github.com/apache/airflow/issues/11329


   **Apache Airflow version**: 1.10.12
   
   **Kubernetes version**: v1.17.6
   
   **Environment**:
   
   - **OS**: CentOS 7
   
   **Context**
   
   I am dynamically launching KubernetesPodOperators within a subdag :
   
   
![image](https://user-images.githubusercontent.com/69032583/95350773-d37f2400-08c0-11eb-9e87-7ebcc0e0be60.png)
   
   Plus, the number n of tasks (gps_statistics_0 .... gps_statistics_n) varies 
with each execution date.
   
   **What happened**:
   
   As you can see on the previous image, all execution dates seem to have 
launched 16 executors. BUT this is not the case. In fact the webserver only 
show the tasks with the same name as the ones in the last execution date (here 
it means tasks gps_statistics_0 .... gps_statistics_n).
   (Which mean if I decided to suffix task's names with the execution date I 
would only see tasks for the last execution date.)
   
   Now if I go to **browse -> task instances** and filter for 
gps_statistics_17, for instance, I can found some tasks with this name, which 
prove that those tasks exist and have been executed.
   
   Sadly it is not just a display issue, because if I try to access the 
gps_statistics_0 instance or log, I get the following error:
   
   
![image](https://user-images.githubusercontent.com/69032583/95350474-7aaf8b80-08c0-11eb-998a-f4e668eac3b9.png)
   
   Furthermore, you can see on the first image that one execution date had only 
6 instances. So at that moment tasks with n > 6 were not available to see or 
click. However when the next execution start, with n >= 16 task with id >6 and 
< n will be back, available to see and click.
   
   So the tasks exist somewhere and just seems to be unavailable at the moment 
of the executions with n inferior to their id.
   
   How is that and is there a way to prevent it ?
   
   Regards


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to