yash1thsa commented on issue #55747: URL: https://github.com/apache/airflow/issues/55747#issuecomment-3362522863
@rawwar / @Ferdinanddb - I was able to debug this issue. Root cause : Synchronous calls triggered underneath async functions. Main issue is in Triggerer code that calls the hook. Triggerer is async by default. But when it hits the hook, the error happens during the initialization itself. Alternatives tried: 1. Create a asynchronous path in trigerrer - get_request_url_header_params, get_connection, get_header etc. But this still did not stop the async to sync issue. 2. Decouple hook from trigger. Handle the hook completely in operator class itself. And send only the connection parameters to trigger. This solves the deadlock . Here is a working code (just for reference not a fix) - [](https://github.com/apache/airflow/compare/main...yash1thsa:airflow:snowflake_async_rc?expand=1 ) Next steps: - Need guidance on the direction . If its ok, need some help on clean refactoring Successful log - `[2025-10-02, 13:31:36] INFO - Generating a new token because the present time (2025-10-02 18:31:36.062698+00:00) is later than the renewal time (2025-10-02 18:31:36.062689+00:00): source="airflow.providers.snowflake.utils.sql_api_generate_jwt" [2025-10-02, 13:31:36] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.hooks.snowflake_sql_api.SnowflakeSqlApiHook" [2025-10-02, 13:31:36] INFO - Deferring task using CustomSnowflakeSqlApiTrigger to avoid AsyncToSync error.: source="airflow.task.operators.airflow.providers.snowflake.operators.snowflake.SnowflakeSqlApiOperator" [2025-10-02, 13:31:36] INFO - Connection Retrieved 'snowflake_test': source="airflow.hooks.base" [2025-10-02, 13:31:36] INFO - DEBUG: Using synchronous JWT path for headers.: source="airflow.providers.snowflake.hooks.snowflake_sql_api.SnowflakeSqlApiHook" [2025-10-02, 13:31:36] INFO - Creating JWTGenerator with arguments account : iqnwtlc-zp51632, user : AIRFLOW_USER, lifetime : 0:59:00, renewal_delay : 0:54:00: source="airflow.providers.snowflake.utils.sql_api_generate_jwt" [2025-10-02, 13:31:36] INFO - Generating a new token because the present time (2025-10-02 18:31:36.539343+00:00) is later than the renewal time (2025-10-02 18:31:36.539338+00:00): source="airflow.providers.snowflake.utils.sql_api_generate_jwt" [2025-10-02, 13:31:36] INFO - Successfully retrieved Authorization header using hook.get_headers().: source="airflow.task.operators.airflow.providers.snowflake.operators.snowflake.SnowflakeSqlApiOperator" [2025-10-02, 13:31:36] INFO - Pausing task as DEFERRED. : dag_id="test_snowflake_dag": task_id="snow": run_id="manual__2025-10-02T18:31:31.969628+00:00": source="task" [2025-10-02, 13:31:37] INFO - trigger test_snowflake_dag/manual__2025-10-02T18:31:31.969628+00:00/snow/-1/1 (ID 51) starting [2025-10-02, 13:31:37] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:38] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:43] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:43] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:48] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:48] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:53] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:54] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:59] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:31:59] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:32:04] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:32:05] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:32:10] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:32:10] INFO - Snowflake SQL GET statements status API response: {'code': '333334', 'message': 'Asynchronous execution in progress. Use provided query id to perform query monitoring and management.', 'statementHandle': '01bf7237-0003-9836-0009-c8660002606a', 'statementStatusUrl': '/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a'}: source="airflow.providers.snowflake.triggers.snowflake_trigger.CustomSnowflakeSqlApiTrigger" [2025-10-02, 13:32:15] INFO - Checking status for query ID 01bf7237-0003-9836-0009-c8660002606a at https://iqnwtlc-zp51632.snowflakecomputing.com/api/v2/statements/01bf7237-0003-9836-0009-c8660002606a: source="airflow.providers.snowflake.triggers.snowflake_tri` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
