GitHub user aledsage opened a pull request:
https://github.com/apache/brooklyn-server/pull/922
Sync on jcloudsLocation.vmInstanceIds
`JcloudsMaxConcurrencyStubbedTest` demonstrates the need for this
synchronization: tearDown can fail because the `obtain` is called in a
different thread from the `releaseMachine`. The id added to `vmInstanceIds` in
obtain was not yet visible to the thread executing `releaseMachine`, so it
failed.
To reproduce, try setting `@Test(invocationCount=...)` to something big.
For me, it failed after 200ish runs.
```
2017-12-20 16:24:31,348 INFO TESTNG INVOKING CONFIGURATION: "Surefire
test" - @BeforeMethod
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.setUp()
2017-12-20 16:24:31,349 INFO Added external config supplier named
'brooklyn-demo-sample':
org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier@67a84d80
2017-12-20 16:24:31,349 INFO TESTNG PASSED CONFIGURATION: "Surefire test"
- @BeforeMethod
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.setUp()
finished in 1 ms
2017-12-20 16:24:31,349 INFO TESTNG INVOKING: "Surefire test" -
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.testConcurrentCreateCalls()
2017-12-20 16:24:31,359 INFO Creating VM aws-ec2:us-east-1 in
JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,360 WARN Cannot check imageChooser status for
aws-ec2:us-east-1 due to manually supplied black-box TemplateBuilder; it is
recommended to use a PortableTemplateBuilder if you supply a TemplateBuilder
2017-12-20 16:24:31,362 INFO Creating VM aws-ec2:us-east-1 in
JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,363 INFO Creating VM aws-ec2:us-east-1 in
JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,363 WARN Cannot check imageChooser status for
aws-ec2:us-east-1 due to manually supplied black-box TemplateBuilder; it is
recommended to use a PortableTemplateBuilder if you supply a TemplateBuilder
2017-12-20 16:24:31,363 INFO Waiting in
JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n] for
machine-creation permit (0 other queuing requests already)
2017-12-20 16:24:31,366 INFO Default SSH keys not found or not usable;
will create new keys for each machine. Create ~/.ssh/id_rsa or set
privateKeyFile / privateKeyPassphrase / password as appropriate for this
location if you wish to be able to log in without Brooklyn.
2017-12-20 16:24:31,366 INFO Default SSH keys not found or not usable;
will create new keys for each machine. Create ~/.ssh/id_rsa or set
privateKeyFile / privateKeyPassphrase / password as appropriate for this
location if you wish to be able to log in without Brooklyn.
2017-12-20 16:24:31,523 INFO Acquired in
JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n] machine-creation
permit, after waiting 160ms
2017-12-20 16:24:31,524 WARN No context entity found in config or current
task
2017-12-20 16:24:31,524 WARN Cannot check imageChooser status for
aws-ec2:us-east-1 due to manually supplied black-box TemplateBuilder; it is
recommended to use a PortableTemplateBuilder if you supply a TemplateBuilder
2017-12-20 16:24:31,524 WARN No context entity found in config or current
task
2017-12-20 16:24:31,527 INFO Default SSH keys not found or not usable;
will create new keys for each machine. Create ~/.ssh/id_rsa or set
privateKeyFile / privateKeyPassphrase / password as appropriate for this
location if you wish to be able to log in without Brooklyn.
2017-12-20 16:24:31,545 INFO Using host-and-port=173.194.32.123:22 and
user=jenkins when connecting to {id=myid, status=RUNNING, loginPort=22,
privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123],
loginUser=myuser}
2017-12-20 16:24:31,545 INFO Using host-and-port=173.194.32.123:22 and
user=jenkins when connecting to {id=myid, status=RUNNING, loginPort=22,
privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123],
loginUser=myuser}
2017-12-20 16:24:31,552 INFO Creating PortForwardManager(scope=global)
2017-12-20 16:24:31,555 WARN Using DEPRECATED flag OPEN_IPTABLES (will not
be supported in future versions) for
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=dol3orvta0)]
at JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,556 WARN Using DEPRECATED flag OPEN_IPTABLES (will not
be supported in future versions) for
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)]
at JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,558 INFO Finished VM aws-ec2:us-east-1 creation:
[email protected]/173.194.32.123:22 ready after 196ms (semaphore obtained
in 0ms;{image={id=us-east-1/bogus-image, providerId=ebs-image-provider,
name=image, location={scope=REGION, id=us-east-1, description=us-east-1,
parent=aws-ec2}, os={family=ubuntu, arch=paravirtual, version=1.0,
description=ubuntu, is64Bit=true}, description=description, version=1.0,
status=AVAILABLE, loginUser=root, userMetadata={rootDeviceType=ebs}},
hardware={id=supporting-bogus, providerId=t2.micro, processors=[{cores=1.0,
speed=0.1}], ram=1024,
supportsImage=Predicates.and(requiresRootDeviceType(ebs),requiresVirtualizationType(paravirtual),idIn([us-east-1/bogus-image]),Predicates.alwaysTrue())},
location={scope=REGION, id=us-east-1, description=us-east-1, parent=aws-ec2},
options={scriptPresent=true, userMetadata={Name=brooklyn-p19pkv-jenkins-ttta,
brooklyn-user=jenkins}}} template built in 38ms; {id=myid, status=RUNNING,
loginP
ort=22, privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123]}
provisioned in 123ms;
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=dol3orvta0)]
connection usable in 25ms; and os customized in 10ms - point /dev/random to
urandom, open iptables)
2017-12-20 16:24:31,561 INFO Finished VM aws-ec2:us-east-1 creation:
[email protected]/173.194.32.123:22 ready after 201ms (semaphore obtained
in 0ms;{image={id=us-east-1/bogus-image, providerId=ebs-image-provider,
name=image, location={scope=REGION, id=us-east-1, description=us-east-1,
parent=aws-ec2}, os={family=ubuntu, arch=paravirtual, version=1.0,
description=ubuntu, is64Bit=true}, description=description, version=1.0,
status=AVAILABLE, loginUser=root, userMetadata={rootDeviceType=ebs}},
hardware={id=supporting-bogus, providerId=t2.micro, processors=[{cores=1.0,
speed=0.1}], ram=1024,
supportsImage=Predicates.and(requiresRootDeviceType(ebs),requiresVirtualizationType(paravirtual),idIn([us-east-1/bogus-image]),Predicates.alwaysTrue())},
location={scope=REGION, id=us-east-1, description=us-east-1, parent=aws-ec2},
options={scriptPresent=true, userMetadata={Name=brooklyn-p19pkv-jenkins-ttta,
brooklyn-user=jenkins}}} template built in 55ms; {id=myid, status=RUNNING,
loginP
ort=22, privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123]}
provisioned in 109ms;
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)]
connection usable in 25ms; and os customized in 12ms - point /dev/random to
urandom, open iptables)
2017-12-20 16:24:31,564 WARN No context entity found in config or current
task
2017-12-20 16:24:31,580 INFO Using host-and-port=173.194.32.123:22 and
user=jenkins when connecting to {id=myid, status=RUNNING, loginPort=22,
privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123],
loginUser=myuser}
2017-12-20 16:24:31,588 WARN Using DEPRECATED flag OPEN_IPTABLES (will not
be supported in future versions) for
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=mbzvjcfoql)]
at JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,590 INFO Finished VM aws-ec2:us-east-1 creation:
[email protected]/173.194.32.123:22 ready after 228ms (semaphore obtained
in 161ms;{image={id=us-east-1/bogus-image, providerId=ebs-image-provider,
name=image, location={scope=REGION, id=us-east-1, description=us-east-1,
parent=aws-ec2}, os={family=ubuntu, arch=paravirtual, version=1.0,
description=ubuntu, is64Bit=true}, description=description, version=1.0,
status=AVAILABLE, loginUser=root, userMetadata={rootDeviceType=ebs}},
hardware={id=supporting-bogus, providerId=t2.micro, processors=[{cores=1.0,
speed=0.1}], ram=1024,
supportsImage=Predicates.and(requiresRootDeviceType(ebs),requiresVirtualizationType(paravirtual),idIn([us-east-1/bogus-image]),Predicates.alwaysTrue())},
location={scope=REGION, id=us-east-1, description=us-east-1, parent=aws-ec2},
options={scriptPresent=true, userMetadata={Name=brooklyn-p19pkv-jenkins-sx1z,
brooklyn-user=jenkins}}} template built in 39ms; {id=myid, status=RUNNING, logi
nPort=22, privateAddresses=[172.168.10.11], publicAddresses=[173.194.32.123]}
provisioned in 1ms;
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=mbzvjcfoql)]
connection usable in 19ms; and os customized in 7ms - point /dev/random to
urandom, open iptables)
2017-12-20 16:24:31,591 INFO TESTNG PASSED: "Surefire test" -
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.testConcurrentCreateCalls()
finished in 242 ms
2017-12-20 16:24:31,591 INFO TESTNG INVOKING CONFIGURATION: "Surefire
test" - @AfterMethod
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.tearDown()
2017-12-20 16:24:31,591 INFO Releasing machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=dol3orvta0)]
in JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n], instance id
myid
2017-12-20 16:24:31,594 INFO Attempted release of unknown machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=dol3orvta0)]
in JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,595 INFO Released machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=dol3orvta0)]:
total time 3ms (semaphore obtained in 1ms; node destroyed in 0ms)
2017-12-20 16:24:31,595 INFO Attempted release of unknown machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)]
in JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,596 WARN Error releasing machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)];
continuing...
java.lang.IllegalArgumentException: Unknown machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)]
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.release(JcloudsLocation.java:2150)
~[classes/:na]
2017-12-20 16:24:31,597 INFO Releasing machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=mbzvjcfoql)]
in JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n], instance id
myid
2017-12-20 16:24:31,599 INFO Attempted release of unknown machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=mbzvjcfoql)]
in JcloudsLocation[aws-ec2:us-east-1:stub-identity@r94r9yeb3n]
2017-12-20 16:24:31,600 INFO Released machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=mbzvjcfoql)]:
total time 2ms (semaphore obtained in 0ms; node destroyed in 0ms)
2017-12-20 16:24:31,614 INFO TESTNG FAILED CONFIGURATION: "Surefire test"
- @AfterMethod
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.tearDown()
finished in 0 ms
org.apache.brooklyn.util.exceptions.CompoundRuntimeException: Error in
tearDown of class
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest
at
org.apache.brooklyn.location.jclouds.AbstractJcloudsLiveTest.tearDown(AbstractJcloudsLiveTest.java:110)
at
org.apache.brooklyn.location.jclouds.AbstractJcloudsStubbedUnitTest.tearDown(AbstractJcloudsStubbedUnitTest.java:72)
at
org.apache.brooklyn.location.jclouds.JcloudsMaxConcurrencyStubbedTest.tearDown(JcloudsMaxConcurrencyStubbedTest.java:117)
Caused by: java.lang.IllegalArgumentException: Unknown machine
SshMachineLocation[173.194.32.123:[email protected]/173.194.32.123:22(id=fyilkxt1h4)]
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.release(JcloudsLocation.java:2150)
at
org.apache.brooklyn.location.jclouds.AbstractJcloudsLiveTest.releaseMachine(AbstractJcloudsLiveTest.java:181)
at
org.apache.brooklyn.location.jclouds.AbstractJcloudsLiveTest.releaseMachineSafely(AbstractJcloudsLiveTest.java:189)
at
org.apache.brooklyn.location.jclouds.AbstractJcloudsLiveTest.tearDown(AbstractJcloudsLiveTest.java:91)
... 33 more
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aledsage/brooklyn-server
fix-JcloudsMaxConcurrencyStubbedTest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/brooklyn-server/pull/922.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #922
----
commit 8670b23b075f944b27d91e0075bd0f4af1ba9175
Author: Aled Sage <aled.sage@...>
Date: 2017-12-21T10:02:42Z
Sync on jcloudsLocation.vmInstanceIds
----
---
