amoghrajesh commented on code in PR #68133:
URL: https://github.com/apache/airflow/pull/68133#discussion_r3393436982
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -3239,6 +3239,21 @@ state_store:
type: integer
example: "10000"
default: "0"
+ max_value_storage_bytes:
+ description: |
+ Only applicable to MetastoreStoreBackend. Maximum size in bytes that a
single task or asset store
+ value written via the core or execution API can have. Values that
exceed this limit are rejected
Review Comment:
Sounds good, this was a bit outdated, handled it
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/task_store.py:
##########
@@ -66,8 +65,12 @@ def value_is_json_representable(cls, v: JsonValue) ->
JsonValue:
serialized = json.dumps(v, allow_nan=False)
except ValueError:
raise ValueError("value contains non-finite numbers; NaN and Inf
are not JSON representable")
- if len(serialized) > _MAX_SERIALIZED_BYTES:
- raise ValueError(f"value exceeds maximum serialized size of
{_MAX_SERIALIZED_BYTES} bytes")
+ limit = conf.getint("state_store", "max_value_storage_bytes")
+ if limit > 0 and len(serialized) > limit:
+ raise ValueError(
+ f"value exceeds max_value_storage_bytes ({limit}); "
+ "for large payloads configure a custom [state_store] backend"
Review Comment:
Fixed — the error message now says "raise [state_store]
max_value_storage_bytes or set it to 0 to disable the limit". Same change
applied to `TaskStorePatchBody` and `AssetStoreBody`.
---
Drafted-by: Claude Code (Sonnet 4.6) (no human review before posting)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
