We just started seeing this a few days ago after turning on SLA for our tasks (not saying SLA did this, may have been happening before and not noticing), but we have a dag that runs once a hour and we see that 4-5 dag runs are marked running but tasks are not getting scheduled. When we get the SLA alert the action we are doing right now is going to the UI and clicking run on tasks manually; this is only needed for the oldest dag run and the rest recover after that. In the past 3 days this has happened twice to us.
We are running 1.8.2, are there any known jira about this? Don't know scheduler well, what could I do to see why these tasks are getting skipped without manual intervention? Thanks for your time.
