[
https://issues.apache.org/jira/browse/AIRFLOW-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788998#comment-16788998
]
Ultrabug commented on AIRFLOW-1135:
-----------------------------------
Hi [~jackjack10]
I believe I've answered this question, but to put it another way : some of us
use XComs that are relevant only for a few runs but tons of them, which would
fill the database for useless data after a short while.
You can argue that we could use another database storage for this but since
it's also a feature of Airflow I thought it would be legitimate to ask (I've
implemented it myself anyway) for a way to simply clean XComs if you wish to.
Last time I checked (time of this issue), there was no other way to clean them
up programaticaly so one had to go straight to the database to do so. I guess
we can do better hence my question / issue.
> Clearing XCom data from previous executions
> -------------------------------------------
>
> Key: AIRFLOW-1135
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1135
> Project: Apache Airflow
> Issue Type: New Feature
> Components: db
> Reporter: Ultrabug
> Priority: Major
>
> It looks like XCom data from previous executions of tasks are never cleared
> from database and that there's no easy way to do so.
> Since XComs are created by execution_date of a task, when a task heavily uses
> XComs (like 20 000+) we can end up with a large xcom table on the database.
> Is there something I'm missing about cleaning up those previous XComs from
> database (since they're not used anymore) ?
> Would you accept the idea of having a new PR providing a helper to remove
> them ?
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
