manipatnam opened a new issue, #59385: URL: https://github.com/apache/airflow/issues/59385
### Apache Airflow Provider(s) snowflake ### Versions of Apache Airflow Providers apache-airflow-providers-snowflake==6.7.0 ### Apache Airflow version 3.1.4 ### Operating System debian ### Deployment Astronomer ### Deployment details _No response_ ### What happened When using `SnowflakeSqlApiOperator` in deferred mode (`deferrable=True`), the operator generates a new JWT token on every polling attempt instead of reusing the existing token. This causes: 1. **Performance overhead**: Unnecessary token generation on every poll (which can happen every few seconds) 2. **Log pollution**: INFO level logs are generated on every poll, flooding the task logs and making it difficult to track actual task progress ## Current Behavior In deferred mode, when the operator polls for query status, it generates a new JWT token for each poll attempt, even though the previous token is still valid. This results in: - Excessive token generation operations - Cluttered logs with repeated JWT generation messages ### What you think should happen instead The JWT token should be reused across polling attempts until it expires or needs renewal. Token generation should only occur when: - No token exists yet - The token has expired - The token renewal time has been reached ### How to reproduce 1. Create a `SnowflakeSqlApiOperator` with `deferrable=True` 2. Execute a long-running query that requires multiple polls 3. Observe the logs - you'll see JWT token generation logs on every poll 4. Check the task logs - they will be cluttered with JWT generation messages ### Anything else ## Impact - **Performance**: Unnecessary cryptographic operations on every poll - **Log pollution**: INFO level logs on every poll make it difficult to track actual task progress - **Resource waste**: CPU cycles spent on token generation that could be avoided ## Additional Context The token caching mechanism exists but is not being utilized effectively in deferred mode. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
