The GitHub Actions job "Tests" on airflow.git/perf/fastapi-orjson-response-optimization has failed. Run started by GitHub user Arunodoy18 (triggered by potiuk).
Head commit for run: 644fc6a9f2450788fdad99dc7d1c691d379090d4 / Arunodoy18 <[email protected]> Optimize FastAPI endpoints with ORJSONResponse for 2x performance improvement This commit implements ORJSONResponse for high-traffic API endpoints to achieve approximately 2x throughput improvement based on benchmark testing. Related issue: #59129 Problem: FastAPI's default response handling revalidates Pydantic models even after initial validation during model construction, causing performance overhead. Benchmark testing showed standard response handling achieved 5,766 req/sec while ORJSONResponse achieved 15,063 req/sec (~2.6x improvement). Solution: 1. Created ORJSONResponse class in airflow/api_fastapi/common/responses.py - Uses orjson library (included with fastapi[standard]) for fast serialization - Configured with OPT_NON_STR_KEYS, OPT_UTC_Z, OPT_SERIALIZE_NUMPY options - Validates data with Pydantic models before serialization - Skips FastAPI's redundant response model revalidation 2. Updated high-traffic list endpoints to use ORJSONResponse: - /monitor/health - Health check endpoint - /dags - DAG list endpoint (main benchmark target) - /providers - Provider list endpoint - /plugins - Plugin list endpoint - /pools - Pool list endpoint Each endpoint now: - Validates response using Pydantic.model_validate() - Converts to dict using model.model_dump() - Returns ORJSONResponse with dict content - Declares response_model for OpenAPI schema generation - Sets response_class=ORJSONResponse 3. Added comprehensive test coverage in test_responses.py - 11 test cases covering various serialization scenarios - Tests datetime handling, nested structures, None values, booleans, etc. Performance improvements (from benchmarks): - Before: 5,766 req/sec, 0.0137s average latency - After: 15,063 req/sec, 0.0059s average latency - Improvement: ~2.6x throughput, ~57% latency reduction Benefits: - Maintains data validation and type safety with Pydantic - Preserves API contract - no breaking changes - OpenAPI documentation remains accurate - Existing tests continue to pass without modification - Significantly better performance for list endpoints under high load Report URL: https://github.com/apache/airflow/actions/runs/20001425294 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
