[
https://issues.apache.org/jira/browse/RAMPART-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Lazarski resolved RAMPART-427.
-------------------------------------
Resolution: Fixed
Fixed in 2.0.0.
Root cause: RampartMessageData's constructor marked a service as client-side
using a
non-atomic check-then-add of the CLIENT_SIDE parameter. Concurrent first
requests to
a service all observed the parameter as absent, then each added it with
setLocked(true). The first call succeeded and locked it; the others failed
with
"The CLIENT_SIDE parameter is already locked and the value cannot be
overridden",
causing the first requests to a freshly secured service to fail
intermittently under
concurrent load (matching the report's ~3-4 failures per 100 runs).
Fix: the parameter is now added by a synchronized, idempotent helper
(RampartMessageData.addClientSideParameterIfAbsent) that locks on the
AxisService and
re-checks before adding, so it is added at most once regardless of how many
threads
race. The lock is per-service rather than a single global static lock (as in
the
originally attached patch), and the common case - parameter already present -
never
enters the synchronized block, so steady-state message processing stays
lock-free.
Testing: added RampartMessageDataTest with an idempotent case (which alone
fails on
the unfixed code) and a 16-thread concurrent case. The integration test
attached to
this issue was not used - it depended on commons-httpclient 3.x, which is no
longer a
Rampart dependency, and was timing-dependent; the new unit test is
deterministic.
While reviewing the surrounding code, one related robustness fix was also
applied:
non-numeric timestampTTL / timestampMaxSkew values in RampartConfig now raise
a clear
RampartException naming the offending parameter, instead of an unhandled
NumberFormatException.
Verified with a full clean 'mvn verify -Papache-release' across all modules,
including the integration scenarios and the nine policy samples, on
OpenJDK 17, 21 and 25.
Thanks to Todor Mazgalov for the report and the original patch.
(Local commits for this issue: 8ce0aeb0 the race fix, c8071bad the review
follow-up.
2.0.0.md still uncommitted on your side.)
> First requests to secured service are failing with "The CLIENT_SIDE parameter
> is already locked and the value cannot be overridden" error in multi-thread
> execution.
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: RAMPART-427
> URL: https://issues.apache.org/jira/browse/RAMPART-427
> Project: Rampart
> Issue Type: Bug
> Components: rampart-core
> Reporter: Todor Mazgalov
> Assignee: Robert Lazarski
> Priority: Major
> Labels: Patch
> Fix For: 2.0.0
>
> Attachments:
> rampart-core-add-synchronization-for-CLIENT_SIDE-parameter.patch,
> rampart-integration-test-testMultiThreadServiceCall.patch
>
>
> First requests to the RampartMessageData in multi-thread execution are
> failing with error for already locked parameter. This occurs when more than
> one thread is trying to set CLIENT_SIDE param to the service, the first
> thread locks the parameter during the addition and the rest cannot override
> it because it's already locked. I'm providing a patch with test which proves
> that when threads are more than one a race condition is available during the
> execution and due to this condition the defect is hard to be reproduced every
> time. In practise it is failing about 3-4 times for 100 runs (based onto my
> machine).
> {quote}
> {{monospaced}}
> org.apache.axis2.AxisFault: Error in extracting message properties
> at
> org.apache.rampart.handler.RampartSender.invoke(RampartSender.java:76)
> at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:335)
> at org.apache.axis2.engine.Phase.invoke(Phase.java:308)
> at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:250)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:415)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:399)
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:150)
> at
> org.apache.axis2.client.ServiceClient.sendRobust(ServiceClient.java:453)
> at
> org.apache.axis2.client.ServiceClient.sendRobust(ServiceClient.java:434)
> at org.apache.rampart.RampartTest$1.run(RampartTest.java:347)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.rampart.RampartException: Error in extracting message
> properties
> at
> org.apache.rampart.RampartMessageData.<init>(RampartMessageData.java:381)
> at org.apache.rampart.MessageBuilder.build(MessageBuilder.java:61)
> at
> org.apache.rampart.handler.RampartSender.invoke(RampartSender.java:65)
> ... 13 more
> Caused by: org.apache.axis2.AxisFault: The CLIENT_SIDE parameter is already
> locked and the value cannot be overridden.
> at
> org.apache.axis2.description.AxisDescription.addParameter(AxisDescription.java:104)
> at
> org.apache.rampart.RampartMessageData.<init>(RampartMessageData.java:197)
> ... 15 more
> org.apache.axis2.AxisFault: Error in extracting message properties
> at
> org.apache.rampart.handler.RampartSender.invoke(RampartSender.java:76)
> at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:335)
> at org.apache.axis2.engine.Phase.invoke(Phase.java:308)
> at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:250)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:415)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:399)
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:150)
> at
> org.apache.axis2.client.ServiceClient.sendRobust(ServiceClient.java:453)
> at
> org.apache.axis2.client.ServiceClient.sendRobust(ServiceClient.java:434)
> at org.apache.rampart.RampartTest$1.run(RampartTest.java:347)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.rampart.RampartException: Error in extracting message
> properties
> at
> org.apache.rampart.RampartMessageData.<init>(RampartMessageData.java:381)
> at org.apache.rampart.MessageBuilder.build(MessageBuilder.java:61)
> at
> org.apache.rampart.handler.RampartSender.invoke(RampartSender.java:65)
> ... 13 more
> Caused by: org.apache.axis2.AxisFault: The CLIENT_SIDE parameter is already
> locked and the value cannot be overridden.
> at
> org.apache.axis2.description.AxisDescription.addParameter(AxisDescription.java:104)
> at
> org.apache.rampart.RampartMessageData.<init>(RampartMessageData.java:197)
> ... 15 more
> {{monospaced}}
> {quote}
> One possible solution is to be used some synchronization mechanism when the
> parameter is being locked, I'm attaching a patch which uses "synchronized"
> block by static object (I suppose that an another synchronization mechanism
> also can be used). I am not sure whether this parameter really should be set
> per service and if it is not then a synchronization will not be need because
> the parameter will not be created for every call but this can be answered in
> the best way by the Rampart developers.
> rampart-core
> org.apache.rampart.RampartMessageData.java
> {code}
> ...
> if (axisService != null && axisService.getParameter(PARAM_CLIENT_SIDE) !=
> null) {
> this.isInitiator = true; //<---- After the addition of the parameter
> the rest of the threads enter here
> } else {
> this.isInitiator = !msgCtx.isServerSide();
> // TODO if Axis Service is null at this point, do we have to
> // create a dummy one ??
> if (this.isInitiator && axisService != null) {
> Parameter clientSideParam = new Parameter(); //<---- Two or
> more threads enter here because the parameter is still not set
> clientSideParam.setName(PARAM_CLIENT_SIDE);
> clientSideParam.setLocked(true); //<---- Parameter is being
> locked
> msgCtx.getAxisService().addParameter(clientSideParam); //<----
> Threads are trying to add the parameter more than once and an exception is
> generated
> }
> }
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
