potiuk commented on issue #9299: URL: https://github.com/apache/airflow/issues/9299#issuecomment-752240505
I agree with @vikramkoka -> XCOM is not supposed to store big amounts of data and MAX_XCOM_SIZE is the maximum you can expect. Rather than increasing the limits I'd turn that into an issue to verify that the MAX_XCOM_SIZE is consistently applied everywhere. The limitation is not due to limit on the field size it is because of the frequency of accessing and model of accessing the XCOM table by Airflow - allowing for much bigger xcom fields might open up scenarios where different components start throttling each other and the whole system might become unusable. This is a dangerous path to go through. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
