elangelo opened a new pull request, #5264:
URL: https://github.com/apache/solr/pull/5264
# Description
With the Security Manager enabled (the default on Java 21 to 23), the S3
backup repository no longer authenticates with web identity credentials
(`AWS_ROLE_ARN` + `AWS_WEB_IDENTITY_TOKEN_FILE`, as set by EKS IAM Roles for
Service Accounts). The default credentials chain falls back to the next
provider, such as the EC2 instance profile, so backups, restores and LISTBACKUP
get a 403 for the node's role, or silently run as the node's role when it
happens to have access to the bucket.
Since AWS SDK 2.46, Apache HttpClient 5 (`apache5-client`) is the SDK's
default HTTP client. `S3StorageClient` passes an explicit `ApacheHttpClient`
(HttpClient 4) to the S3 client, so S3 calls themselves are fine, but the web
identity provider builds its own STS client with the default, now HttpClient 5.
HttpClient 5.6 sets TCP keep-alive socket options, which need
`jdk.net.NetworkPermission` under a Security Manager:
```
java.lang.IllegalStateException: Apache5HttpClient requires
jdk.net.NetworkPermission for "setOption.TCP_KEEPIDLE",
"setOption.TCP_KEEPINTERVAL", "setOption.TCP_KEEPCOUNT" when a SecurityManager
is active.
Caused by: java.security.AccessControlException: access denied
("jdk.net.NetworkPermission" "setOption.TCP_KEEPIDLE")
```
The AWS SDK documents these permissions in its announcement of the Apache 5
default: https://github.com/aws/aws-sdk-java-v2/issues/7007.
Affected: 9.11.0 (AWS SDK 2.53.1) and 10.1.0 (2.54.6) on Java 21 to 23. Not
affected: 9.10.1 and earlier and 10.0.0 (SDK before 2.46), and Java 24+, where
`bin/solr` leaves the Security Manager off (so the official 10.1.0 Docker
image, on Java 25, is fine).
# Solution
Grant the three permissions in `server/etc/security.policy`, next to the
other entries for the AWS SDK.
The S3 repository tests do not cover this path: they use static credentials
against S3Mock, never the web identity provider.
Workaround for affected versions, until this is released:
`-Dsoftware.amazon.awssdk.http.service.impl=software.amazon.awssdk.http.apache.ApacheSdkHttpService`
in `SOLR_OPTS` makes the SDK use HttpClient 4 for every client.
# Tests
A small class calling
`WebIdentityTokenFileCredentialsProvider.create().resolveCredentials()`, run
with the S3 module's jars and the JVM options `bin/solr` uses with the Security
Manager, against a dummy role and token:
- 10.1.0 distribution on Java 21, with the 10.1.0 `security.policy`: fails
with the `NetworkPermission` error above.
- Same, with the `security.policy` from this branch: gets past the
permission check and calls STS (which rejects the dummy token, as expected).
- 9.11.0 on EKS with a real service account role: with the same three
permissions granted, the provider resolves the role.
On 9.11.0 we have also been running the workaround above on a 6-node
SolrCloud since 2026-10-22: LISTBACKUP, the nightly backups and a full
collection RESTORE use the pod's role again.
This change was developed at ONTOFORCE (https://www.ontoforce.com), which
contributes it to Apache Solr. It was written with the help of an AI coding
assistant (Claude Code) and reviewed by me.
# Checklist
- [x] I have reviewed the guidelines for How to Contribute and my code
conforms to the standards described there to the best of my ability.
- [ ] I have created a Jira issue and added the issue ID to my pull request
title.
- [ ] I have given Solr maintainers access to contribute to my PR branch.
- [x] I have developed this patch against the `main` branch.
- [ ] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the Reference Guide
- [x] I have added a changelog entry for my change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
