[
https://issues.apache.org/jira/browse/BROOKLYN-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754534#comment-15754534
]
Aled Sage commented on BROOKLYN-404:
------------------------------------
I think the "right" solution is to change {{BrooklynProperties}} so that it
does not implement {{java.util.Map}}, and to have the internal map as a private
field. We can then control access to that map much more easily, and write
thread-safe code.
Although it "breaks backwards compatibility", I think that's fine because it's
in the package {{org.apache.brooklyn.core.internal}}. It would only be a
problem if user-code has explicitly cast things to
{{org.apache.brooklyn.core.internal.BrooklynProperties}} or to
{{java.util.Map}}. We can live with that.
[~svet] or anyone else, do you agree with that change?
I've started playing around with this - I'll come back to it when I have some
more time.
p.s. If we didn't care so much about backwards compatibility, I'd also like to
revisit {{StringConfigMap}} and {{ConfigMap}}, but let's not bite off to much
in one go!
> ConcurrentModificationException on rebind to persisted state (in
> BrooklynProperties)
> ------------------------------------------------------------------------------------
>
> Key: BROOKLYN-404
> URL: https://issues.apache.org/jira/browse/BROOKLYN-404
> Project: Brooklyn
> Issue Type: Bug
> Reporter: Aled Sage
>
> With Brooklyn 0.10.0-SNAPSHOT, when attempting to restart Brooklyn (binding
> to existing persisted state), it hit a {{ConcurrentModificationException}}.
> This failure only happens occasionally.
> Starting Brooklyn such that it fails fast if there are any persistence errors:
> {noformat}
> ./bin/brooklyn launch --persist auto --persistenceDir
> `pwd`/$PERSISTENCE_DIR/data --highAvailability disabled
> --startupFailOnPersistenceErrors --startupFailOnManagedAppsErrors
> {noformat}
> The info log showed:
> {noformat}
> 2016-11-24 09:49:31,612 ERROR Startup error: Error rebinding to persisted
> state: Failure rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
> at
> org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:129)
> ~[brooklyn-utils-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure
> rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[na:1.8.0_111]
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure
> rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> at
> org.apache.brooklyn.util.exceptions.Exceptions.create(Exceptions.java:432)
> ~[brooklyn-utils-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> Caused by: java.lang.IllegalStateException: problem rebinding location
> cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)])
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onRebindFailed(RebindExceptionHandlerImpl.java:304)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> Caused by: java.util.ConcurrentModificationException: null
> at
> java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
> ~[na:1.8.0_111]
> Startup error: Error rebinding to persisted state: Failure rebinding: problem
> rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> {noformat}
> The debug log showed:
> {noformat}
> 2016-11-24 09:49:31,610 DEBUG o.a.b.l.common.BasicLauncher [main]: Error
> rebinding to persisted state (rethrowing):
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure
> rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
> at
> org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:129)
> ~[brooklyn-utils-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebind(RebindManagerImpl.java:513)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.launcher.common.BasicLauncher.startPersistenceWithoutHA(BasicLauncher.java:662)
> [brooklyn-launcher-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.launcher.common.BasicLauncher.startPersistence(BasicLauncher.java:624)
> [brooklyn-launcher-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.launcher.common.BasicLauncher.handlePersistence(BasicLauncher.java:511)
> [brooklyn-launcher-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.launcher.common.BasicLauncher.start(BasicLauncher.java:429)
> [brooklyn-launcher-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at org.apache.brooklyn.cli.Main$LaunchCommand.call(Main.java:462)
> [brooklyn-cli-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at org.apache.brooklyn.cli.Main$LaunchCommand.call(Main.java:197)
> [brooklyn-cli-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.cli.AbstractMain.execCli(AbstractMain.java:252)
> [brooklyn-cli-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.cli.AbstractMain.execCli(AbstractMain.java:244)
> [brooklyn-cli-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at io.cloudsoft.amp.Main.main(Main.java:39)
> [amp-launcher-3.5.1-20161121.1656.jar:na]
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure
> rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[na:1.8.0_111]
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> ~[na:1.8.0_111]
> at
> com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
> ~[guava-16.0.1.jar:na]
> at
> org.apache.brooklyn.util.core.task.BasicTask.get(BasicTask.java:361)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebind(RebindManagerImpl.java:511)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> ... 9 common frames omitted
> org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure
> rebinding: problem rebinding location cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)]):
> ConcurrentModificationException
> at
> org.apache.brooklyn.util.exceptions.Exceptions.create(Exceptions.java:432)
> ~[brooklyn-utils-common-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onDoneImpl(RebindExceptionHandlerImpl.java:497)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onDone(RebindExceptionHandlerImpl.java:413)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindIteration.run(RebindIteration.java:268)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebindImpl(RebindManagerImpl.java:558)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl$3.call(RebindManagerImpl.java:508)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl$3.call(RebindManagerImpl.java:506)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:522)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> Caused by: java.lang.IllegalStateException: problem rebinding location
> cMBNYzIx
> (SshMachineLocation[SshMachineLocation:cMBN:[email protected]/54.184.92.166:22(id=cMBNYzIx)])
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onRebindFailed(RebindExceptionHandlerImpl.java:304)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindIteration.reconstructEverything(RebindIteration.java:543)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindIteration.doRun(RebindIteration.java:243)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.InitialFullRebindIteration.doRun(InitialFullRebindIteration.java:69)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindIteration.run(RebindIteration.java:266)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> ... 8 common frames omitted
> Caused by: java.util.ConcurrentModificationException: null
> at
> java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
> ~[na:1.8.0_111]
> at
> java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:752)
> ~[na:1.8.0_111]
> at
> java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:750)
> ~[na:1.8.0_111]
> at
> org.apache.brooklyn.core.internal.BrooklynPropertiesImpl.getAllConfigLocalRaw(BrooklynPropertiesImpl.java:463)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.internal.BrooklynPropertiesImpl.getAllConfig(BrooklynPropertiesImpl.java:470)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.internal.DeferredBrooklynProperties.asMapWithStringKeys(DeferredBrooklynProperties.java:162)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.BasicLocationRegistry.getProperties(BasicLocationRegistry.java:516)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.AbstractLocationResolver.extractConfig(AbstractLocationResolver.java:101)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.access.PortForwardManagerLocationResolver.extractConfig(PortForwardManagerLocationResolver.java:84)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.access.PortForwardManagerLocationResolver.newLocationSpecFromString(PortForwardManagerLocationResolver.java:60)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.BasicLocationRegistry.getLocationSpec(BasicLocationRegistry.java:375)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.BasicLocationRegistry.getLocationSpec(BasicLocationRegistry.java:355)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.location.BasicLocationRegistry.getLocationManaged(BasicLocationRegistry.java:331)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.location.ssh.SshMachineLocation.init(SshMachineLocation.java:306)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.BasicLocationRebindSupport.addCustoms(BasicLocationRebindSupport.java:108)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.BasicLocationRebindSupport.addCustoms(BasicLocationRebindSupport.java:38)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.AbstractBrooklynObjectRebindSupport.reconstruct(AbstractBrooklynObjectRebindSupport.java:65)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> at
> org.apache.brooklyn.core.mgmt.rebind.RebindIteration.reconstructEverything(RebindIteration.java:541)
> ~[brooklyn-core-0.10.0-20161121.1656.jar:0.10.0-20161121.1656]
> ... 11 common frames omitted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
