+1. I am all for failing it. I do not see an immediate issue with it - especially if we also explain how to handle it (i.e. custom XCom, limiting data, truncating data).
I assume this is only when "standard" XCom is used. I would be for it, explicit is better than implicit. The XCom limit has always been a bit "vague" - we wrote something alongside "not too big" value and you could check in the code what the limit is but we never explicitly specified that as a limitation, but it was there. For me this looks like even more details of something that we already started discussing separately - i.e. being explicit about the "public interface" of ours and being explicit with that. If our aim is being an "extensible platform", then we have to be very clear about what our users can and cannot do with Airflow. J. On Tue, Dec 20, 2022 at 4:03 AM Ping Zhang <[email protected]> wrote: > Hi community, > > I would like to discuss if we want to fail early for Xcom.set when the > value exceeds the storage limit. Otherwise, the database (default xcom > backend) truncates the value leading to failure when Xcom reads it, (e.g. > on UI, Admin -> Xcoms will also fail). > > But I am not sure whether we want to fail the whole task execution if > Xcom.set fails and would like to have your inputs. > > Thanks, > > Ping >
