adoroszlai opened a new pull request, #6089:
URL: https://github.com/apache/ozone/pull/6089
## What changes were proposed in this pull request?
Revoked S3 secret is stored in the cache as `S3SecretValue` with `null`
secret value. Since the object is mutable, other thread could access the same
instance before revocation, and might encounter NPE while trying to convert to
protobuf.
```
2024-01-24 09:49:09,063 [om1-OMDoubleBufferFlushThread] ERROR
ratis.OzoneManagerDoubleBuffer (ExitUtils.java:terminate(133)) - Terminating
with exit status 2: During flush to DB encountered error in OMDoubleBuffer
flush thread om1-OMDoubleBufferFlushThread
org.apache.ratis.util.ExitUtils$ExitException: During flush to DB
encountered error in OMDoubleBuffer flush thread om1-OMDoubleBufferFlushThread
when handling OMRequest: cmdType: SetS3Secret
traceID: ""
success: true
status: OK
SetS3SecretResponse {
accessId: "testUser1accessId1"
secretKey: "testsecret"
}
at org.apache.ratis.util.ExitUtils.terminate(ExitUtils.java:141)
at org.apache.ratis.util.ExitUtils.terminate(ExitUtils.java:151)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.terminate(OzoneManagerDoubleBuffer.java:559)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatch(OzoneManagerDoubleBuffer.java:413)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushBatch(OzoneManagerDoubleBuffer.java:345)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushCurrentBuffer(OzoneManagerDoubleBuffer.java:319)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushTransactions(OzoneManagerDoubleBuffer.java:286)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$S3Secret$Builder.setAwsSecret(OzoneManagerProtocolProtos.java)
at
org.apache.hadoop.ozone.om.helpers.S3SecretValue.getProtobuf(S3SecretValue.java:100)
at
org.apache.hadoop.hdds.utils.db.DelegatedCodec.toCodecBuffer(DelegatedCodec.java:85)
at
org.apache.hadoop.hdds.utils.db.Codec.toDirectCodecBuffer(Codec.java:73)
at
org.apache.hadoop.hdds.utils.db.TypedTable.putWithBatch(TypedTable.java:171)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl$1.addWithBatch(OmMetadataManagerImpl.java:1949)
at
org.apache.hadoop.ozone.om.response.s3.security.OMSetSecretResponse.addToDBBatch(OMSetSecretResponse.java:80)
at
org.apache.hadoop.ozone.om.response.OMClientResponse.checkAndUpdateDB(OMClientResponse.java:66)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.lambda$9(OzoneManagerDoubleBuffer.java:407)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatchWithTrace(OzoneManagerDoubleBuffer.java:244)
at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatch(OzoneManagerDoubleBuffer.java:406)
... 4 more
```
was seen in `TestMultiTenantVolume` (4 / 1000 runs).
This PR changes `S3SecretValue` to be immutable.
https://issues.apache.org/jira/browse/HDDS-10200
## How was this patch tested?
`TestMultiTenantVolume` passed 10x100:
https://github.com/adoroszlai/ozone/actions/runs/7645630287
Regular CI:
https://github.com/adoroszlai/ozone/actions/runs/7645622076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
