rhenar0 opened a new issue, #13563: URL: https://github.com/apache/cloudstack/issues/13563
### problem When several users log in via Google OAuth2 at the same time (e.g. a classroom of students connecting at the start of a lab session), some logins fail with `Unable to verify the email address with the provided secret`, and then all subsequent Google OAuth2 logins fail until `cloudstack-management` is restarted. From the end user's perspective, the OAuth2 SSO looks like it "disabled itself". `GoogleOAuth2Provider` is a singleton Spring bean, but it caches the OAuth `accessToken` / `refreshToken` in shared mutable instance fields, making the login flow non-thread-safe. [https://github.com/apache/cloudstack/blob/main/plugins/user-authenticators/oauth2/src/main/java/org/apache/cloudstack/oauth2/google/GoogleOAuth2Provider.java](https://github.com/apache/cloudstack/blob/main/plugins/user-authenticators/oauth2/src/main/java/org/apache/cloudstack/oauth2/google/GoogleOAuth2Provider.java) ``` public class GoogleOAuth2Provider extends AdapterBase implements UserOAuth2Authenticator { protected String accessToken = null; protected String refreshToken = null; ... @Override public String verifyCodeAndFetchEmail(String secretCode) { ... if (StringUtils.isAnyEmpty(accessToken, refreshToken)) { GoogleTokenResponse tokenResponse = flow.newTokenRequest(secretCode) .setRedirectUri(redirectURI) .execute(); accessToken = tokenResponse.getAccessToken(); refreshToken = tokenResponse.getRefreshToken(); } GoogleCredential credential = ... .setAccessToken(accessToken).setRefreshToken(refreshToken); Userinfo userinfo = oauth2.userinfo().get().execute(); return userinfo.getEmail(); } ``` And in `verifyUser()`: ``` String verifiedEmail = verifyCodeAndFetchEmail(secretCode); if (verifiedEmail == null || !email.equals(verifiedEmail)) { throw new CloudRuntimeException("Unable to verify the email address with the provided secret"); } clearAccessAndRefreshTokens(); // <-- only reached on the success path ``` Two compounding problems: 1. Race condition between concurrent logins. The tokens are shared by all in-flight logins. If user A's tokens are stored and user B enters `verifyCodeAndFetchEmail()` before A reaches `clearAccessAndRefreshTokens()`, the `isAnyEmpty(...)` guard is false, so B skips exchanging its own authorization code and calls userinfo with A's tokens. Google returns A's email, which does not match B's, so `verifyUser()` throws. 2. Poisoned state is never cleaned up. `clearAccessAndRefreshTokens()` is only called on the success path of `verifyUser()`. Any failure (the email mismatch above, or an `IOException` on the `userinfo` call) leaves the stale tokens in the singleton. From then on, every subsequent login skips its own code exchange and reuses the stale/foreign token: - while the stolen access token is still valid, `userinfoù returns the original user's email → email mismatch → all other users fail; - once it expires (and the refresh token is invalid/not usable), `userinfo` fails with 401 → everyone fails. The only way to reset the singleton's fields is to restart the management server, which matches the observed behavior exactly. Note that the token caching provides no benefit at all: each login carries its own one-time authorization code that must be exchanged individually. The instance fields are simply a bug. ### versions CloudStack 4.22.0 Management server: Rocky Linux, single management server Hypervisors: KVM Configuration: oauth2.enabled = true, Google registered as OAuth2 provider (client ID / secret / redirect URI), used as the primary login method for a school environment (~hundreds of users) ### The steps to reproduce the bug 1. Enable OAuth2 (oauth2.enabled = true) and register Google as a provider. 2. Have several distinct users (distinct Google accounts, each mapped to a CloudStack user) initiate the Google OAuth2 login flow concurrently (a handful of simultaneous logins is enough; the more concurrency, the easier it triggers.) 3. Observe some logins failing with Unable to verify the email address with the provided secret. 4. After the first such failure, try any new Google OAuth2 login (any user, any browser): it fails too. All Google OAuth2 logins keep failing until cloudstack-management is restarted. The race can also be demonstrated deterministically in a debugger by pausing one thread between the token assignment and `clearAccessAndRefreshTokens()` while a second thread runs `verifyCodeAndFetchEmail()`. ### What to do about it? Remove the shared instance fields and use local variables. Every call must exchange its own authorization code: ``` @Override public String verifyCodeAndFetchEmail(String secretCode) { ... GoogleTokenResponse tokenResponse; try { tokenResponse = flow.newTokenRequest(secretCode) .setRedirectUri(redirectURI) .execute(); } catch (IOException e) { throw new CloudRuntimeException(String.format("Failed to exchange the authorization code: %s", e.getMessage())); } String accessToken = tokenResponse.getAccessToken(); // local String refreshToken = tokenResponse.getRefreshToken(); // local GoogleCredential credential = new GoogleCredential.Builder() .setTransport(httpTransport) .setJsonFactory(jsonFactory) .setClientSecrets(clientSecrets) .build() .setAccessToken(accessToken) .setRefreshToken(refreshToken); ... } ``` And delete the accessToken/refreshToken fields together with `clearAccessAndRefreshTokens()` (also removing its call site in verifyUser()). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
