The GitHub Actions job "Tests AMD" on airflow.git/v3-1-test has failed.
Run started by GitHub user kaxil (triggered by kaxil).

Head commit for run:
d9969be6c9b44e9c3bbccf7dc5e9e0c9d06db734 / Kaxil Naik <[email protected]>
Decouple serialization and deserialization code for tasks (#54569)

Remove Task SDK dependencies from airflow-core deserialization by establishing
a schema-based contract between client and server components. This
change enables independent deployment and upgrades while laying the foundation
for multi-language SDK support.

Key Decoupling Achievements:
- Replace dynamic get_serialized_fields() calls with hardcoded class methods
- Add schema-driven default resolution with get_operator_defaults_from_schema()
- Remove OPERATOR_DEFAULTS import dependency from airflow-core
- Implement SerializedBaseOperator class attributes for all operator defaults
- Update _is_excluded() logic to use schema defaults for efficient serialization

Serialization Optimizations:
- Unified partial_kwargs optimization supporting both encoded/non-encoded 
formats
- Intelligent default exclusion reducing storage redundancy
- MappedOperator.operator_class memory optimization (~90-95% reduction)
- Comprehensive client_defaults system with hierarchical resolution

Compatibility & Performance:
- Significant size reduction for typical DAGs with mapped operators
- Minimal overhead for client_defaults section (excellent efficiency)
- All existing serialized DAGs continue to work unchanged

Technical Implementation:
- Add generate_client_defaults() with LRU caching for optimal performance
- Implement _deserialize_partial_kwargs() supporting dual formats
- Centralized field deserialization eliminating code duplication
- Consolidated preprocessing logic in _preprocess_encoded_operator()
- Callback field preprocessing for backward compatibility

Testing & Validation:
- Added TestMappedOperatorSerializationAndClientDefaults with 9 comprehensive 
tests
- Parameterized tests for multiple serialization formats
- End-to-end validation of serialization/deserialization workflows
- Backward compatibility validation for callback field migration

This decoupling enables independent deployment/upgrades and provides the
foundation for multi-language SDK ecosystem alongside the Task Execution API.

Part of https://github.com/apache/airflow/issues/45428

Report URL: https://github.com/apache/airflow/actions/runs/17337104226

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to