shubhransh-eb commented on issue #33647: URL: https://github.com/apache/airflow/issues/33647#issuecomment-1831178128
@QuintenBruynseraede , From what I observe, every min triggerer runs a query to find the number of triggers that are running at a given time. If query doesnt return anything in 1 min, it tries 5 time and then it kills the pod of triggerer and this is a continuous cycle, that is why your liveness probes checks are failing. Before running analyze command we set `STATS_SAMPLE_PAGES` (its for mysql) as well to solve the issue. From what we figured out, airflow databases added a triggerer id to task where it is required, but when task is done it removes that id maying that column value as null. Now if you have large table size (3-4M+) there are max 15-20 rows with that column and hence when you run analyze command it doesnt help much because cardinality is still off. We increase page size so that it scans more rows to find cardinality and hence temporarily solves the issue. other than this, we are trying to keep size of table to be around 3M by daily deleting old entries. Now sure if these 2 steps can help you as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
