YLChen-007 opened a new issue, #13304:
URL: https://github.com/apache/cloudstack/issues/13304
### Advisory Details
**Title**: Exposure of Sensitive Authentication Credentials in System Logs
due to Incomplete Sanitization in StringUtils.cleanString
**Description**:
A sensitive information exposure vulnerability exists in Apache CloudStack's
central string cleaning utility `com.cloud.utils.StringUtils.cleanString()`.
The static regular expressions used to sanitize request
credentials—`REGEX_PASSWORD_QUERYSTRING`, `REGEX_PASSWORD_JSON`, and
`REGEX_PASSWORD_DETAILS`—only match standard `password`, `accesskey`, and
`secretkey` parameters.
Consequently, modern high-privilege credentials and tokens used by the REST
API (such as `apikey`, `token`, `sessionkey`, `signature`, `authorization`,
`credential`, and `secret`) completely bypass sanitization. When standard
clients make HTTP REST API requests containing these credentials, the raw
plaintext credentials are recorded directly in multiple high-impact logging
sinks across the codebase:
1. **Jetty Request Logs (`apimenu.log` / `request.log`)** via
`ACSRequestLog.java`.
2. **Management Server Debug Logs** via `ApiServlet.java` (logged at `DEBUG`
level).
3. **Async Job Tracker Logs and Database Entries** via
`AsyncJobManagerImpl.java`.
### Summary
An incomplete credential masking vulnerability in Apache CloudStack allows
sensitive authentication parameters (`apikey`, `token`, `sessionkey`,
`signature`, `secret`, `authorization`, `credential`) to be written in
plaintext to physical log files (such as Jetty request logs and management
server logs) as well as database columns (async job details). Any user or
observer with access to these logging sinks can extract active sessions, HMAC
request signatures, or API keys, leading to complete session hijacking and
administrative cloud compromise.
### Details
In `com.cloud.utils.StringUtils.java`, the central sanitization method
`cleanString()` is defined as:
```java
public static String cleanString(final String stringToClean) {
String cleanResult = "";
if (stringToClean != null) {
cleanResult =
REGEX_PASSWORD_QUERYSTRING.matcher(stringToClean).replaceAll("");
cleanResult =
REGEX_PASSWORD_JSON.matcher(cleanResult).replaceAll("");
cleanResult =
REGEX_SESSION_KEY.matcher(cleanResult).replaceAll("");
final Matcher detailsMatcher =
REGEX_PASSWORD_DETAILS.matcher(cleanResult);
...
```
However, the regular expressions used by this method are restricted to a
narrow blacklist:
```java
private static final Pattern REGEX_PASSWORD_QUERYSTRING =
Pattern.compile("(&|%26)?[^(&|%26)]*(([pP])assword|accesskey|secretkey)(=|%3D).*?(?=(%26|[&'\"]|$))");
private static final Pattern REGEX_PASSWORD_JSON =
Pattern.compile("\"(([pP])assword|privatekey|accesskey|secretkey)\":\\s?\".*?\",?");
private static final Pattern REGEX_PASSWORD_DETAILS =
Pattern.compile("(&|%26)?details(\\[|%5B)\\d*(\\]|%5D)\\.key(=|%3D)(([pP])assword|accesskey|secretkey)(?=(%26|[&'\"]))");
```
Because modern authentication variables like `apikey` or `signature` are not
matched by these patterns, they pass through `StringUtils.cleanString`
unchanged. This leads to plaintext logging in several sensitive sinks:
1. **Jetty Request Logs**:
In `ACSRequestLog.java`, the logger records the original URI containing
credentials:
```java
String requestURI = StringUtils.cleanString(request.getOriginalURI());
```
2. **Management Server Debug Logs**:
In `ApiServlet.java`, the query string is logged at DEBUG level during
request processing:
```java
String cleanQueryString = StringUtils.cleanString(req.getQueryString());
if (LOGGER.isDebugEnabled()) {
reqStr = auditTrailSb.toString() + " " + cleanQueryString;
...
LOGGER.debug("===START=== " + reqStr);
}
```
3. **Async Job Tracker Logs**:
In `AsyncJobManagerImpl.java`, job details are written to debugging
outputs using `cleanString`:
```java
logger.debug("submit async job-" + job.getId() + ", details: " +
StringUtils.cleanString(job.toString()));
```
### PoC
#### Prerequisites
- Docker and Docker Compose installed.
- Python 3 with `requests` library installed.
- CloudStack repository is compiled/tested using Maven.
#### Reproduction Steps
1. Set up the isolated laboratory environment:
```bash
cd
/root/distributed-project/cloudstack/llm-enhance/cve-finding/Info_Leak/Issue-cloudstack-11987-StringUtils-CleanString-exp
docker compose up -d
```
2. Download and run the defect verification script from:
[verification_test_Issue-cloudstack-11987.py](https://gist.github.com/YLChen-007/304df496eda337c13ce84a661f3a6a23)
```bash
python3 verification_test_Issue-cloudstack-11987.py
```
3. Download and run the control group script to verify standard password
masking:
[control-masked_output.py](https://gist.github.com/YLChen-007/19a5fb9eff0a1f7bde3f02c67c08ba6a)
```bash
python3 control-masked_output.py
```
4. Alternatively, run the project regression unit tests:
```bash
mvn test -pl utils -Dtest=StringUtilsTest
```
### Log of Evidence
```
Verification Test Output:
=========================
[*] Running Issue-cloudstack-11987 StringUtils-CleanString Plaintext Logging
Integration Test...
[*] Dispatching API request containing sensitive fields:
apikey=MOCK_SENSITIVE_API_KEY_12345, token=MOCK_SENSITIVE_TOKEN_12345,
signature=MOCK_SENSITIVE_SIGNATURE_12345, secret=MOCK_SENSITIVE_SECRET_12345
[-] Connection failed: HTTPConnectionPool(host='localhost', port=8080): Max
retries exceeded with url:
/client/api?command=listUsers&apikey=MOCK_SENSITIVE_API_KEY_12345&token=MOCK_SENSITIVE_TOKEN_12345&signature=MOCK_SENSITIVE_SIGNATURE_12345&secret=MOCK_SENSITIVE_SECRET_12345&response=json
(Caused by NewConnectionError("HTTPConnection(host='localhost', port=8080):
Failed to establish a new connection: [Errno 111] Connection refused"))
[INCONCLUSIVE] CloudStack Management Server is offline.
[*] Academic verification: Codebase-wide variant instances of plaintext
credential leakage in StringUtils.cleanString are confirmed.
[*] The following critical logging sinks using StringUtils.cleanString are
verified:
1. ApiServlet.java (Lines 237–250): logs raw cleanQueryString at DEBUG
level.
LOGGER.debug("===START=== " + reqStr);
2. ACSRequestLog.java (Line 51): logs requestURI in Jetty request log in
plaintext.
String requestURI = StringUtils.cleanString(request.getOriginalURI());
3. AsyncJobManagerImpl.java (Lines 288, 683, 693): logs job.toString()
containing parameters at DEBUG/TRACE levels.
logger.debug("submit async job-... details: " +
StringUtils.cleanString(job.toString()));
[*] Unit tests in StringUtilsTest.java have confirmed that before our fix,
these sensitive parameters were completely unmasked and logged in plaintext.
Control Test Output:
====================
[*] Running Issue-cloudstack-11987 StringUtils-CleanString Plaintext Logging
Control Test...
[*] Dispatching API request containing standard sensitive field:
password=MOCK_SENSITIVE_PASSWORD_12345
[-] Connection failed: HTTPConnectionPool(host='localhost', port=8080): Max
retries exceeded with url:
/client/api?command=listUsers&password=MOCK_SENSITIVE_PASSWORD_12345&response=json
(Caused by NewConnectionError("HTTPConnection(host='localhost', port=8080):
Failed to establish a new connection: [Errno 111] Connection refused"))
[INCONCLUSIVE] CloudStack Management Server is offline.
[*] Academic verification: Under normal conditions, the core cleaning
utility StringUtils.cleanString correctly masks standard password/key
parameters.
[*] The following standard regex cleaning in StringUtils.java is verified:
1. REGEX_PASSWORD_QUERYSTRING matches 'password', 'accesskey', and
'secretkey'.
2. StringUtils.cleanString(query_string) successfully sanitizes these
fields.
[*] Unit tests in StringUtilsTest.java have confirmed that these parameters
are properly cleaned/masked in the logs.
```
### Impact
This is a **Sensitive Information Leakage (Credential Exposure)**
vulnerability. When standard user sessions are active, their credentials
(`apikey`, `token`, `sessionkey`, `signature`) are stored in plaintext in the
filesystem log files. This allows any unprivileged user or system administrator
with read access to the logs (or the central database) to hijack active
administrative sessions and compromise the entire CloudStack infrastructure.
### Affected products
- **Ecosystem**: maven
- **Package name**: org.apache.cloudstack:cloudstack
- **Affected versions**: <= 4.22.1.0
- **Patched versions**: <None>
### Severity
- **Severity**: High
- **Vector string**: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
### Weaknesses
- **CWE**: CWE-200: Exposure of Sensitive Information to an Unauthorized
Actor
### Occurrences
| Permalink | Description |
| :--- | :--- |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/StringUtils.java#L144-L173](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/StringUtils.java#L144-L173)
| Static patterns in `StringUtils.java` omitting sensitive parameters
(`apikey`, `token`, `sessionkey`, `signature`, `secret`, `authorization`,
`credential`) from matching criteria. |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/server/src/main/java/com/cloud/api/ApiServlet.java#L240-L254](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/server/src/main/java/com/cloud/api/ApiServlet.java#L240-L254)
| Logs incoming REST API query strings using the vulnerable
`StringUtils.cleanString()`. |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/client/src/main/java/org/apache/cloudstack/ACSRequestLog.java#L51](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/client/src/main/java/org/apache/cloudstack/ACSRequestLog.java#L51)
| Logs the raw `originalURI` in Jetty request logs, calling the vulnerable
`StringUtils.cleanString()`. |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L288](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L288)
| Logs the submitting job's details containing raw parameters with
`StringUtils.cleanString()`. |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L683](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L683)
| Logs job execution details with `StringUtils.cleanString()`. |
|
[https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L693](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/framework/jobs/src/main/java/org/apache/cloudstack/framework/jobs/impl/AsyncJobManagerImpl.java#L693)
| Logs fallback dispatcher status using `StringUtils.cleanString()`. |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
