HuanjieGuo commented on issue #46856: URL: https://github.com/apache/airflow/issues/46856#issuecomment-2668979782
> [@HuanjieGuo](https://github.com/HuanjieGuo) > > This might help > > > The DAG drops the archived tables it created in the cleanup process by default using the --skip-archive option and does not maintain any history. If the task fails for example if it runs for longer than five minutes, the archive tables are not cleared. By running airflow db drop-archived in the second task of the DAG, we ensure all archive tables are dropped even in the event of the first task failing. > > https://www.astronomer.io/docs/learn/cleanup-dag-tutorial/#warnings > > the cli has [drop archive](https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#drop-archived) command -> `airflow db drop-archived -y` When we run `airflow db clean `, we open the --skip-archive parameter. We cannot get the failure from the command line to know which table element cleaning is failed. So the failure archive tables will accumulate. Why don't we just delete the archive table if user already set the --skip-archive when we met any failure. It does not require further operation from user side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
