dstandish edited a comment on pull request #19324: URL: https://github.com/apache/airflow/pull/19324#issuecomment-968316195
> The boto3 secrets manager library will fail with ResourceNotFoundException if there are no restrictions in the IAM role, i.e. it has full access to secrets manager. That isn't practical for most organizations, as they will have several entities accessing secrets manager and do not want to give full access to all secrets by all of them. OK so what you're saying here is that it's `not practical for most organizations` to catch `ResourceNotFoundException` because in most organizations they'll get `AccessDeniedException` instead, because the cred exists but the instance does not have permission to access it. But why is the airflow instance trying to retrieve the cred that it does not have access to in the first place? That seems like a misconfiguration issue. If the scheduler is trying to access, for example the value for `sql_alchemy_conn` from secrets backend (that's your scenario right?), and it is unable to do so, isn't the scheduler going to fail anyway? You have to jump through some real hoops to make secrets backend retrieve config keys from secrets backend. And this might not be such a good thing. But this reality means that if you're trying to retrieve config from secrets backend, you should really expect your cluster to have access to that secret. Please help me understand. I will make time to be more responsive to this one going forward so we can make sure to get to resolution soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
