[jira] [Commented] (JCLOUDS-1132) Amazon AWS - Create Volume - query params reported as unrecognized
[
https://issues.apache.org/jira/browse/JCLOUDS-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637060#comment-16637060
]
Aled Sage commented on JCLOUDS-1132:
TL;DR: Is it safe to bump the API version to "2016-11-15" in AWSEC2ApiMetadata?
I successfully tested with \{{Encrypted=1}}, which is in the examples at
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateVolume.html#API_CreateVolume_Examples
To do this, you also need to bump the version in
org.jclouds.aws.ec2.AWSEC2ApiMetadata (e.g. to "2016-11-15", from "2012-06-01").
Without the version bump, you still get the error shown below:
{noformat}
j.wire [ager-Z59Bm9lQ-45] >>
"Action=CreateVolume=eu-west-1b=4=1=gp2=2012-06-01"
j.headers [ager-Z59Bm9lQ-45] >> POST https://ec2.eu-west-1.amazonaws.com/
HTTP/1.1
j.headers [ager-Z59Bm9lQ-45] >> Host: ec2.eu-west-1.amazonaws.com
j.headers [ager-Z59Bm9lQ-45] >> X-Amz-Date: 20181003T135559Z
j.headers [ager-Z59Bm9lQ-45] >> Authorization: AWS4-HMAC-SHA256
Credential=/20181003/eu-west-1/ec2/aws4_request,
SignedHeaders=content-type;host;x-amz-date,
Signature=
j.headers [ager-Z59Bm9lQ-45] >> Content-Type: application/x-www-form-urlencoded
j.headers [ager-Z59Bm9lQ-45] >> Content-Length: 100
j.headers [ager-Z59Bm9lQ-45] << HTTP/1.1 400 Bad Request
j.headers [ager-Z59Bm9lQ-45] << Transfer-Encoding: chunked
j.headers [ager-Z59Bm9lQ-45] << Server: AmazonEC2
j.headers [ager-Z59Bm9lQ-45] << Connection: close
j.headers [ager-Z59Bm9lQ-45] << Date: Wed, 03 Oct 2018 13:55:58 GMT
j.headers [ager-Z59Bm9lQ-45] << Content-Type: application/unknown
j.wire [ager-Z59Bm9lQ-45] << "[\n]"
j.wire [ager-Z59Bm9lQ-45] <<
"UnknownParameterThe parameter
Encrypted is not
recognized1da4ed2f-112b-49ab-867e-ef56f5656f72"
{noformat}
For the \{{RunInstances}} with \{{BlockDeviceMapping}}, I think it's right to
use \{{BlockDeviceMapping.1.Ebs.Encrypted=true}}, but you also need a version
like "2016-11-15".
I didn't create an AMI with an encrypted EBS boot volume (see
https://aws.amazon.com/blogs/aws/new-encrypted-ebs-boot-volumes/), so my test
didn't work end-to-end, but the error I got back was sensible:
{noformat}
j.wire [nager-aju8Oj09-5] >>
"Action=RunInstances=ami-3548444c=1=1=t2.micro=sg-0c764025773c2701e=jclouds%23brooklyn-pg12a0-aledsage-ebs-e-jdye-vanillasoftwar-ufot%237c6=I2Nsb3VkLWNvbmZpZwpyZXBvX3VwZ3JhZGU6IG5vbmUK=/dev/sda1=8=true=gp2=true=2016-11-15"
j.wire [nager-aju8Oj09-5] <<
"InvalidBlockDeviceMappingthe
encrypted flag cannot be specified since device /dev/sda1 has a snapshot
specified.da2e8338-c2c2-4c47-95fe-3b476fc51bef"
{noformat}
Here are my code diffs to make this use-case work (against jclouds 2.1.0):
{noformat}
--- a/providers/aws-ec2/src/main/java/org/jclouds/aws/ec2/AWSEC2ApiMetadata.java
+++ b/providers/aws-ec2/src/main/java/org/jclouds/aws/ec2/AWSEC2ApiMetadata.java
@@ -62,7 +62,7 @@ public final class AWSEC2ApiMetadata extends
BaseHttpApiMetadata {
public static final class Builder extends
BaseHttpApiMetadata.Builder {
public Builder() {
id("aws-ec2")
- .version("2012-06-01")
+ .version("2016-11-15")
diff --git
a/apis/ec2/src/main/java/org/jclouds/ec2/options/CreateVolumeOptions.java
b/apis/ec2/src/main/java/org/jclouds/ec2/options/CreateVolumeOptions.java
index 9c2e1821ad..388da7cd43 100644
--- a/apis/ec2/src/main/java/org/jclouds/ec2/options/CreateVolumeOptions.java
+++ b/apis/ec2/src/main/java/org/jclouds/ec2/options/CreateVolumeOptions.java
@@ -92,7 +92,7 @@ public class CreateVolumeOptions extends
BaseEC2RequestOptions {
*/
public CreateVolumeOptions isEncrypted(boolean encrypted) {
if (encrypted)
- formParameters.put("Encrypted", "true");
+ formParameters.put("Encrypted", "1");
return this;
}
diff --git
a/apis/ec2/src/test/java/org/jclouds/ec2/options/CreateVolumeOptionsTest.java
b/apis/ec2/src/test/java/org/jclouds/ec2/options/CreateVolumeOptionsTest.java
index 20f74754ca..88ca91688f 100644
---
a/apis/ec2/src/test/java/org/jclouds/ec2/options/CreateVolumeOptionsTest.java
+++
b/apis/ec2/src/test/java/org/jclouds/ec2/options/CreateVolumeOptionsTest.java
@@ -137,7 +137,7 @@ public class CreateVolumeOptionsTest {
CreateVolumeOptions options = new CreateVolumeOptions();
options.isEncrypted(true);
assertEquals(options.buildFormParameters().get("Encrypted"),
- ImmutableList.of("true"));
+ ImmutableList.of("1"));
}
@@ -150,7 +150,7 @@ public class CreateVolumeOptionsTest {
public void testIsEncryptedStatic() {
CreateVolumeOptions options = isEncrypted(true);
assertEquals(options.buildFormParameters().get("Encrypted"),
- ImmutableList.of("true"));
+ ImmutableList.of("1"));
}
{noformat}
> Amazon AWS - Create Volume - query params reported as unrecognized
> --
>
> Key: JCLOUDS-1132
> URL:
[jira] [Commented] (JCLOUDS-1132) Amazon AWS - Create Volume - query params reported as unrecognized
[
https://issues.apache.org/jira/browse/JCLOUDS-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474406#comment-16474406
]
Aled Sage commented on JCLOUDS-1132:
This still seems to be a problem for me with jclouds 2.1.0. I set
{{volumeOptions.isEncrypted(true)}}, and it failed with:
{noformat}
org.jclouds.aws.AWSResponseException: request POST
https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='----67dc264118f0', requestToken='null',
code='UnknownParameter', message='The parameter Encrypted is not recognized',
context='{Response=, Errors=}'}{noformat}
---
The {{volumeType}} works for me with {{gp2}} and {{io1}}.
> Amazon AWS - Create Volume - query params reported as unrecognized
> --
>
> Key: JCLOUDS-1132
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1132
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 1.9.0, 1.9.2
> Environment: Amazon AWS EC2 management console
>Reporter: Tusa Mihail Cristian
>Priority: Major
>
> Let say that I want to create a Volume that is encrypted. So, I set the
> encrypted on "true"... in request query params apear "=true". In
> this case the error that I got is:
> UnknownParameterThe parameter
> Encrypted is not
> recognizedc874434a-b00e-425d-baf4-18087441603a
>
> instead it should be: "=1" as it is shown in Amazon docs:
> http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateVolume.html
> in Example/Sample request. The problems are also related with Volume Type. If
> not specified than creates a standard volume but if anything specified (any
> of valid Values: standard | io1 | gp2 | sc1 | st1 ) the same error with
> unrecognized param is received.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (JCLOUDS-1417) EC2 describeVolumes wrong when tags present
Aled Sage created JCLOUDS-1417:
--
Summary: EC2 describeVolumes wrong when tags present
Key: JCLOUDS-1417
URL: https://issues.apache.org/jira/browse/JCLOUDS-1417
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 2.1.0
Reporter: Aled Sage
When calling \{{elasticBlockStoreApi.describeVolumesInRegion(region,
volumeId)}}, it fails to parse the response correctly if the volume has tags.
For example, when it gets the XML response below:
{noformat}
http://ec2.amazonaws.com/doc/2012-06-01/;>
95d47b6d-5f7a-44a2-922d-03be76b73c0a
vol-01234567890123456
1
eu-west-1a
available
2018-05-14T08:52:45.448Z
mykey
myval
gp2
100
{noformat}
It returns:
{noformat}
[
Volume [attachments=[], availabilityZone=eu-west-1a, createTime=Mon May 14
09:52:45 BST 2018, id=vol-01234567890123456, region=us-east-1, size=1,
snapshotId=null, status=available, volumeType=null, iops=null,
encrypted=false],
Volume [attachments=[], availabilityZone=null, createTime=null, id=null,
region=us-east-1, size=0, snapshotId=null, status=null, volumeType=gp2,
iops=100, encrypted=false]]
]
{noformat}
This is presumably because \{{DescribeVolumesResponseHandler.endElement}} looks
to see if it was the "item" element, and if so adds that volume. This causes it
to record the half-parsed volume when it comes across the tag's item, and then
to treat the rest of the volume's definition as a second volume.
You can easily see this by adding tags to
{{apis/ec2/src/test/resources/describe_volumes.xml}}, and looking at the
results of {{DescribeVolumesResponseHandlerTest}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (JCLOUDS-1381) NullPointerException provisioning VM in GCE: could not find image for disk
Aled Sage created JCLOUDS-1381:
--
Summary: NullPointerException provisioning VM in GCE: could not
find image for disk
Key: JCLOUDS-1381
URL: https://issues.apache.org/jira/browse/JCLOUDS-1381
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 2.0.2
Reporter: Aled Sage
Using jclouds 2.0.2, when attempting to provision a VM in GCE it failed with
the exception shown below. However, my retry immediately after this did succeed.
{noformat}
com.google.common.util.concurrent.UncheckedExecutionException:
java.util.concurrent.ExecutionException: could not find image for disk
https://www.googleapis.com/compute/v1/projects/cloudsoft-qa/zones/europe-west1-b/disks/qa-kubernetes-xlgv2n5qsb-312:
null
at
com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4832)
at
org.jclouds.googlecomputeengine.compute.functions.InstanceToNodeMetadata.apply(InstanceToNodeMetadata.java:80)
at
org.jclouds.googlecomputeengine.compute.functions.InstanceToNodeMetadata.apply(InstanceToNodeMetadata.java:44)
at
com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
at com.google.common.collect.Iterators$8.transform(Iterators.java:799)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:651)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators.indexOf(Iterators.java:776)
at com.google.common.collect.Iterators.any(Iterators.java:684)
at com.google.common.collect.Iterables.any(Iterables.java:623)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.getNextNames(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:199)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:123)
at
org.jclouds.googlecomputeengine.compute.strategy.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:130)
at
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:217)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:756)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:614)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:451)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:440)
at org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:113)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:421)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:396)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:364)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:565)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: could not find image for
disk
https://www.googleapis.com/compute/v1/projects/cloudsoft-qa/zones/europe-west1-b/disks/qa-kubernetes-xlgv2n5qsb-312:
null
at
org.jclouds.googlecomputeengine.compute.loaders.DiskURIToImage.load(DiskURIToImage.java:53)
at
org.jclouds.googlecomputeengine.compute.loaders.DiskURIToImage.load(DiskURIToImage.java:33)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
at
com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)
... 28 more
Caused by: java.lang.NullPointerException
at
[jira] [Created] (JCLOUDS-1378) NullPointerException in NovaComputeServiceAdapter when VM fails to start
Aled Sage created JCLOUDS-1378:
--
Summary: NullPointerException in NovaComputeServiceAdapter when VM
fails to start
Key: JCLOUDS-1378
URL: https://issues.apache.org/jira/browse/JCLOUDS-1378
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 2.0.3
Reporter: Aled Sage
When provisioning a VM in Openstack Nova, I hit the \{{NullPointerException}}
shown below:
{noformat}
createNodesInGroup(qa-amp1-xwbbiuo4qv) java.lang.NullPointerException
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210)
com.google.common.base.Splitter.split(Splitter.java:386)
org.jclouds.openstack.nova.v2_0.compute.NovaComputeServiceAdapter.createNodeWithGroupEncodedIntoName(NovaComputeServiceAdapter.java:141)
org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.createNodeWithGroupEncodedIntoName(AdaptingComputeServiceStrategies.java:196)
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet$AddNode.call(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:81)
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet$AddNode.call(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:64)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
java.lang.Thread.getStackTrace(Thread.java:1559)
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.createNodeInGroupWithNameAndTemplate(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:169)
org.jclouds.openstack.nova.v2_0.compute.strategy.ApplyNovaTemplateOptionsCreateNodesWithGroupEncodedIntoNameThenAddToSet.createNodeInGroupWithNameAndTemplate(ApplyNovaTemplateOptionsCreateNodesWithGroupEncodedIntoNameThenAddToSet.java:183)
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:124)
org.jclouds.openstack.nova.v2_0.compute.strategy.ApplyNovaTemplateOptionsCreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(ApplyNovaTemplateOptionsCreateNodesWithGroupEncodedIntoNameThenAddToSet.java:160)
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:217)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)
com.sun.proxy.$Proxy180.createNodesInGroup:-1)
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:756)
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:614)
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:451)
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:440)
org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:113)
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:421)
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:396)
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:364)
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:565)
java.util.concurrent.FutureTask.run(FutureTask.java:266)}]},"extendedStackTrace":[{"class":"com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
{noformat}
Earlier in the log, it showed:
{noformat}
2018-01-24T14:09:40.517Z : {"timeMillis":1516802980478,"thread":"user thread
1","level":"DEBUG","loggerName":"jclouds.compute","message":">> creating new
server region(ca-ymq-1) name(qa-amp2-uiz49vcvcc-cd7)
image(f99752dc-8b85-4aca-837b-c3109da4b1ad)
flavor(52a01f6b-e660-48b5-8c06-5fb2a0fab0ec)
options(CreateServerOptions{keyName=opensta
ck,
[jira] [Closed] (JCLOUDS-1283) IllegalArgumentException in OpenStack destroyNode, after failure to provision
[
https://issues.apache.org/jira/browse/JCLOUDS-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aled Sage closed JCLOUDS-1283.
--
Resolution: Duplicate
Thanks [~nacx] - you're right, it's a duplicate. Closing.
> IllegalArgumentException in OpenStack destroyNode, after failure to provision
> -
>
> Key: JCLOUDS-1283
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1283
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 1.9.2
>Reporter: Aled Sage
>Priority: Trivial
>
> As reported in https://issues.apache.org/jira/browse/BROOKLYN-295, with
> jclouds 1.9.2 (looking at the line numbers, it looks like 2.0.1 would have
> the same behaviour).
> I attempted to deploy an app to OpenStack. It failed to provision due to
> insufficient resources (hence tricky to reproduce). But then jclouds tried
> and failed to destroy the VM.
> It would be nice if jclouds handled the failure more gracefully. This
> exception hides the message about the provisioning request failing.
> The exception in jclouds was:
> {noformat}
> java.lang.IllegalArgumentException: id must be in format regionId/id
> {noformat}
> The provisioning error (shown in the OpenStack web-console) was:
> {noformat}
> Message No valid host was found. Exceeded max scheduling attempts 3 for
> instance e74952b7-f495-4148-9a09-c4dee0d75e7a. Last exception: [u'Traceback
> (most recent call last):\n', u' File
> "/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/c
> Code 500
> Details File
> "/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/conductor/manager.py",
> line 669, in build_instances instances[0].uuid) File
> "/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/scheduler/utils.py",
> line 172, in populate_retry raise exception.NoValidHost(reason=msg)
> Created June 8, 2016, 10:33 a.m.
> {noformat}
> The Brooklyn debug log shows:
> {noformat}
> 2016-06-08 11:33:45,215 DEBUG o.a.b.l.j.JcloudsLocation
> [brooklyn-execmanager-yA1Lh5GF-504]: jclouds using template
> {image={id=RegionOne/e16a7bca-7363-45b1-bd6a-02479d4cea77,
> providerId=e16a7bca-7363-45b1-bd6a-02479d4cea77, name=CentOS 7,
> location={scope=REGION, id=RegionOne, description=RegionOne,
> parent=openstack-nova}, os={family=centos, name=CentOS 7, version=,
> description=CentOS 7, is64Bit=true}, description=CentOS 7, status=AVAILABLE,
> loginUser=root}, hardware={id=RegionOne/2, providerId=2, name=m1.small,
> location={scope=REGION, id=RegionOne, description=RegionOne,
> parent=openstack-nova}, processors=[{cores=1.0, speed=1.0}], ram=2048,
> volumes=[{type=LOCAL, size=20.0, bootDevice=true, durable=true}],
> supportsImage=ALWAYS_TRUE}, location={scope=REGION, id=RegionOne,
> description=RegionOne, parent=openstack-nova}, options={loginUser=centos,
> loginPrivateKeyPresent=true, inboundPorts=[22, 3376, 2376],
> scriptPresent=true, securityGroups=[VPN_local],
> userMetadata={Name=brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29-zj5s,
> brooklyn-user=aled, brooklyn-app-id=yfxfed97j6, brooklyn-app-name=Docker
> Swarm, brooklyn-entity-id=rj29jfrfvy, brooklyn-entity-name=swarm-node,
> brooklyn-server-creation-date=2016-06-08-1133},
> networks=[e472fd1f-01f8-493a-b284-b0ff0b0e305f], autoAssignFloatingIp=false,
> keyPairName=openstack, configDrive=false}} / options {loginUser=centos,
> loginPrivateKeyPresent=true, inboundPorts=[22, 3376, 2376],
> scriptPresent=true, securityGroups=[VPN_local],
> userMetadata={Name=brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29-zj5s,
> brooklyn-user=aled, brooklyn-app-id=yfxfed97j6, brooklyn-app-name=Docker
> Swarm, brooklyn-entity-id=rj29jfrfvy, brooklyn-entity-name=swarm-node,
> brooklyn-server-creation-date=2016-06-08-1133},
> networks=[e472fd1f-01f8-493a-b284-b0ff0b0e305f], autoAssignFloatingIp=false,
> keyPairName=openstack, configDrive=false} to provision machine in
> openstack-nova:https:/acme.com:5000/v2.0@VanillaSoftwareProcessImpl{id=rj29jfrfvy}
> 2016-06-08 11:33:45,215 DEBUG o.a.b.l.j.JcloudsLocation
> [brooklyn-execmanager-yA1Lh5GF-504]: NOTE: unused flags passed to obtain VM
> in
> openstack-nova:https://acme.com:5000/v2.0@VanillaSoftwareProcessImpl{id=rj29jfrfvy}:
> {jclouds.keystone.credential-type=,
> machineCreationSemaphore=java.util.concurrent.Semaphore@78d9e87b[Permits =
> 2147483643]}
> 2016-06-08 11:33:45,215 DEBUG jclouds.compute
> [brooklyn-execmanager-yA1Lh5GF-504]: >> running 1 node
> group(brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29)
> location(RegionOne) image(RegionOne/e16a7bca-7363-45b1-bd6a-02479d4cea77)
> hardwareProfile(RegionOne/2) options({loginUser=centos,
> loginPrivateKeyPresent=true, inboundPorts=[22, 3376,
[jira] [Created] (JCLOUDS-1283) IllegalArgumentException in OpenStack destroyNode, after failure to provision
Aled Sage created JCLOUDS-1283:
--
Summary: IllegalArgumentException in OpenStack destroyNode, after
failure to provision
Key: JCLOUDS-1283
URL: https://issues.apache.org/jira/browse/JCLOUDS-1283
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 1.9.2
Reporter: Aled Sage
Priority: Trivial
As reported in https://issues.apache.org/jira/browse/BROOKLYN-295, with jclouds
1.9.2 (looking at the line numbers, it looks like 2.0.1 would have the same
behaviour).
I attempted to deploy an app to OpenStack. It failed to provision due to
insufficient resources (hence tricky to reproduce). But then jclouds tried and
failed to destroy the VM.
It would be nice if jclouds handled the failure more gracefully. This exception
hides the message about the provisioning request failing.
The exception in jclouds was:
{noformat}
java.lang.IllegalArgumentException: id must be in format regionId/id
{noformat}
The provisioning error (shown in the OpenStack web-console) was:
{noformat}
Message No valid host was found. Exceeded max scheduling attempts 3 for
instance e74952b7-f495-4148-9a09-c4dee0d75e7a. Last exception: [u'Traceback
(most recent call last):\n', u' File
"/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/c
Code 500
Details File
"/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/conductor/manager.py",
line 669, in build_instances instances[0].uuid) File
"/opt/bbc/openstack-11.0-bbc173/nova/local/lib/python2.7/site-packages/nova/scheduler/utils.py",
line 172, in populate_retry raise exception.NoValidHost(reason=msg)
Created June 8, 2016, 10:33 a.m.
{noformat}
The Brooklyn debug log shows:
{noformat}
2016-06-08 11:33:45,215 DEBUG o.a.b.l.j.JcloudsLocation
[brooklyn-execmanager-yA1Lh5GF-504]: jclouds using template
{image={id=RegionOne/e16a7bca-7363-45b1-bd6a-02479d4cea77,
providerId=e16a7bca-7363-45b1-bd6a-02479d4cea77, name=CentOS 7,
location={scope=REGION, id=RegionOne, description=RegionOne,
parent=openstack-nova}, os={family=centos, name=CentOS 7, version=,
description=CentOS 7, is64Bit=true}, description=CentOS 7, status=AVAILABLE,
loginUser=root}, hardware={id=RegionOne/2, providerId=2, name=m1.small,
location={scope=REGION, id=RegionOne, description=RegionOne,
parent=openstack-nova}, processors=[{cores=1.0, speed=1.0}], ram=2048,
volumes=[{type=LOCAL, size=20.0, bootDevice=true, durable=true}],
supportsImage=ALWAYS_TRUE}, location={scope=REGION, id=RegionOne,
description=RegionOne, parent=openstack-nova}, options={loginUser=centos,
loginPrivateKeyPresent=true, inboundPorts=[22, 3376, 2376], scriptPresent=true,
securityGroups=[VPN_local],
userMetadata={Name=brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29-zj5s,
brooklyn-user=aled, brooklyn-app-id=yfxfed97j6, brooklyn-app-name=Docker Swarm,
brooklyn-entity-id=rj29jfrfvy, brooklyn-entity-name=swarm-node,
brooklyn-server-creation-date=2016-06-08-1133},
networks=[e472fd1f-01f8-493a-b284-b0ff0b0e305f], autoAssignFloatingIp=false,
keyPairName=openstack, configDrive=false}} / options {loginUser=centos,
loginPrivateKeyPresent=true, inboundPorts=[22, 3376, 2376], scriptPresent=true,
securityGroups=[VPN_local],
userMetadata={Name=brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29-zj5s,
brooklyn-user=aled, brooklyn-app-id=yfxfed97j6, brooklyn-app-name=Docker Swarm,
brooklyn-entity-id=rj29jfrfvy, brooklyn-entity-name=swarm-node,
brooklyn-server-creation-date=2016-06-08-1133},
networks=[e472fd1f-01f8-493a-b284-b0ff0b0e305f], autoAssignFloatingIp=false,
keyPairName=openstack, configDrive=false} to provision machine in
openstack-nova:https:/acme.com:5000/v2.0@VanillaSoftwareProcessImpl{id=rj29jfrfvy}
2016-06-08 11:33:45,215 DEBUG o.a.b.l.j.JcloudsLocation
[brooklyn-execmanager-yA1Lh5GF-504]: NOTE: unused flags passed to obtain VM in
openstack-nova:https://acme.com:5000/v2.0@VanillaSoftwareProcessImpl{id=rj29jfrfvy}:
{jclouds.keystone.credential-type=,
machineCreationSemaphore=java.util.concurrent.Semaphore@78d9e87b[Permits =
2147483643]}
2016-06-08 11:33:45,215 DEBUG jclouds.compute
[brooklyn-execmanager-yA1Lh5GF-504]: >> running 1 node
group(brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29)
location(RegionOne) image(RegionOne/e16a7bca-7363-45b1-bd6a-02479d4cea77)
hardwareProfile(RegionOne/2) options({loginUser=centos,
loginPrivateKeyPresent=true, inboundPorts=[22, 3376, 2376], scriptPresent=true,
securityGroups=[VPN_local],
userMetadata={Name=brooklyn-o8g809-aled-docker-swarm-yfxf-swarm-node-rj29-zj5s,
brooklyn-user=aled, brooklyn-app-id=yfxfed97j6, brooklyn-app-name=Docker Swarm,
brooklyn-entity-id=rj29jfrfvy, brooklyn-entity-name=swarm-node,
brooklyn-server-creation-date=2016-06-08-1133},
networks=[e472fd1f-01f8-493a-b284-b0ff0b0e305f], autoAssignFloatingIp=false,
[jira] [Created] (JCLOUDS-1236) Provisioning of CentOS 7.3 VM can leave sshd in bad state
Aled Sage created JCLOUDS-1236:
--
Summary: Provisioning of CentOS 7.3 VM can leave sshd in bad state
Key: JCLOUDS-1236
URL: https://issues.apache.org/jira/browse/JCLOUDS-1236
Project: jclouds
Issue Type: Bug
Components: jclouds-scriptbuilder
Affects Versions: 2.0.0
Reporter: Aled Sage
Priority: Minor
On provisioning CentOS 7.3, use of
{{org.jclouds.scriptbuilder.statements.ssh.SshdConfig}} to call {{service sshd
reload}} can (often) leave systemd in a bad state. It causes systemd to think
that the process is "inactive (dead)".
This code is called by
{{org.jclouds.scriptbuilder.statements.login.AdminAccess}} when {{lockSsh}} is
true (which is the default).
For a lot more detail, see:
* https://issues.apache.org/jira/browse/BROOKLYN-434, which describes the
behaviour we see.
* RHEL bug report at https://bugzilla.redhat.com/show_bug.cgi?id=1381997 (which
gives the best description of the underlying problem in
https://bugzilla.redhat.com/show_bug.cgi?id=1381997#c4).
* CentOS bug report at https://bugs.centos.org/view.php?id=12757
* The change introduced the problematic behavior at
https://bugzilla.redhat.com/show_bug.cgi?id=1291172
A workaround in jclouds would be:
{noformat}
- Statement reloadSshdConfig = exec("hash service 2>&- && service ssh
reload 2>&- || service sshd reload 2>&- || /etc/init.d/ssh* reload");
+ Statement reloadSshdConfig = exec("( hash service 2>&- && ( service ssh
reload || ( service sshd stop; service sshd start ) ) ) || /etc/init.d/ssh*
reload");
{noformat}
---
However, I'd hope that the CentOS/RHEL etc community will fix this in sshd
and/or systemd. Once that is available, then hopefully {{service sshd reload}}
will work (and is the right, clean thing to do).
But even if that happens, there will likely still be a bunch of images in
various clouds that have this bug.
We could conclude that we live with the bug (perhaps mentioning it as a known
issue in the release notes, or just via this bug report), and rely on it being
fixed in a future CentOS version.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (JCLOUDS-1230) azurecompute-arm: VM deletion takes very long time (many extra api calls?)
Aled Sage created JCLOUDS-1230:
--
Summary: azurecompute-arm: VM deletion takes very long time (many
extra api calls?)
Key: JCLOUDS-1230
URL: https://issues.apache.org/jira/browse/JCLOUDS-1230
Project: jclouds
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Aled Sage
Deleting a VM in azurecompute-arm took 5mins 55seconds. Looking at the logging
for that thread, I suspect there are a lot of optimisations that could be done.
Here's a rough breakdown of the destroy steps, and how long they take:
* Get details of the VM and other pieces (e.g. network interface, etc
3 seconds
* List all the images
11 seconds
* List all the storageaccounts
3mins 39secs
* List providers/locations
1 second
* Delete the VM, and poll for completion .
1min 51secs
* Delete nic
3 seconds
* Delete VM disk storage
1 second
* delete storage account
4 seconds
Can we avoid listing all the images and all the storage?
Below are some log snippets that illustrate this (all taken from a single
thread):
{noformat}
2017-01-25 16:01:44,266 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> destroying
node(southeastasia/qa-entity-id-ca)
...
2017-01-25 16:01:44,267 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking GetVirtualMachine
...
2017-01-25 16:01:47,859 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking sku:list
...
2017-01-25 16:01:58,598 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking resourcegroup:list
...
2017-01-25 16:01:58,628 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking storageaccount:list
...
2017-01-25 16:05:37,543 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking providers:get
...
2017-01-25 16:05:37,592 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking location:list
...
...
2017-01-25 16:05:38,916 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> destroying
southeastasia/qa-entity-id-ca ...
2017-01-25 16:05:38,917 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking DeleteVirtualMachine
...
2017-01-25 16:07:29,170 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> destroying
nic jc-nic-qa-entity-id-ca...
2017-01-25 16:07:29,171 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking networkinterfacecard:delete
...
2017-01-25 16:07:32,091 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> deleting
public ip nic public-address-qa-entity-id-ca...
2017-01-25 16:07:32,091 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking publicipaddress:delete
...
2017-01-25 16:07:34,107 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> deleting
virtual machine disk storage...
2017-01-25 16:07:34,121 DEBUG 101 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking DeleteContainer
...
2017-01-25 16:07:35,141 DEBUG 106 j.compute [ger-M9fVKLUG-247] >> deleting
storage account qaentityid524b...
2017-01-25 16:07:35,142 DEBUG 106 o.j.r.i.InvokeHttpMethod [ger-M9fVKLUG-247]
>> invoking storageaccount:delete
...
2017-01-25 16:07:39,855 DEBUG 106 j.compute [ger-M9fVKLUG-247] << destroyed
node(southeastasia/qa-entity-id-ca) success(true)
{noformat}
Also, combined with https://issues.apache.org/jira/browse/JCLOUDS-1229 this
could be very dangerous: we might well be rate-limited while trying to list all
the images and thus the VM deletion / cleanup would be aborted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JCLOUDS-1229) Azure ARM - No retry is the rate limit is exceeded
[
https://issues.apache.org/jira/browse/JCLOUDS-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838067#comment-15838067
]
Aled Sage commented on JCLOUDS-1229:
[~nacx] I'd be inclined to go with the existing default (i.e. max two minutes
per backoff) - i.e. to trust azure to not do anything too crazy for how long we
are told to back-off for. It's better to be slow and successful, than for
jclouds to abort during "normal" azure rate-limiting. My reasoning is that a
user of jclouds can always cntrl-c or wrap the call in a timeout of their own.
Also the existing default (i.e. no retry) means we risk aborting vm-deletion
when half-way through, which could leave behind the nic, public-ip or VM. It
becomes much harder for the user of jclouds to retry, because they would need
to inspect exception details to know if it was safe to retry.
For the publishers, I think that in aws-ec2 we default to a pre-defined set of
image-owners when listing AMIs (but ignore that list of owners if an explicit
imageId or an image-owner override is specified). We could/should do something
similar for azure-arm, making it configurable for which publishers to include.
([~andreaturli] I expect you'll be interested in this conversation.)
> Azure ARM - No retry is the rate limit is exceeded
> --
>
> Key: JCLOUDS-1229
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1229
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute, jclouds-labs
>Affects Versions: 2.0.0
>Reporter: Thomas Bouron
> Labels: azurecompute-arm
>
> Trying to provision a VM in Azure ARM. My deployment failed because of the
> rate limit exceeded with the stacktrace below.
> I was expecting jclouds to perform a back-off retry as it does for other
> clouds but does not seem to be the case (See {{AzureRateLimitRetryHandler}} -
> is that wired in correctly, and does it correctly handle 429 response code)?
> There were 390 API calls made in 2mins 22 secs before getting this exception.
> {noformat}
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking resourcegroup:list
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request 300674855: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups?api-version=2015-01-01
> HTTP/1.1
> 2017-01-25 11:57:04,729 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:04,729 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response 300674855: HTTP/1.1 200 OK
> 2017-01-25 11:57:04,732 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking storageaccount:list
> 2017-01-25 11:57:04,733 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request -641560959: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups/aleks-test-res/providers/Microsoft.Storage/storageAccou
> 2017-01-25 11:57:04,763 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,264 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,764 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,938 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:05,938 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response -641560959: HTTP/1.1 429
> 2017-01-25 11:57:05,940 WARN 126 o.a.b.l.j.JcloudsLocation
> [nager-XBhutgvo-4] Error loading available images to report (following
> original error matching template which will be rethrown):
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTT
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:73)
> at
>
[jira] [Commented] (JCLOUDS-1229) Azure ARM - No retry is the rate limit is exceeded
[
https://issues.apache.org/jira/browse/JCLOUDS-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837926#comment-15837926
]
Aled Sage commented on JCLOUDS-1229:
[~nacx] My personal preference would be for the default in jclouds to be "to
work, even if a little slowly". Retrying when rate-limited sounds like a good
default.
We saw 17 seconds as the rate-limit time, which is surprisingly high!
There's config for {{jclouds.max-ratelimit-wait}} (defaulting to 2 minutes) to
try to avoid things taking crazy-long. But then if you're rate-limited for 17
seconds every few requests, and jclouds executes approx 390 requests, then I do
agree that can add up to considerable delays.
It would be nice to reduce the number of API calls if at all possible (e.g. can
we get details of multiple images in a single api call?).
> Azure ARM - No retry is the rate limit is exceeded
> --
>
> Key: JCLOUDS-1229
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1229
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute, jclouds-labs
>Affects Versions: 2.0.0
>Reporter: Thomas Bouron
> Labels: azurecompute-arm
>
> Trying to provision a VM in Azure ARM. My deployment failed because of the
> rate limit exceeded with the stacktrace below.
> I was expecting jclouds to perform a back-off retry as it does for other
> clouds but does not seem to be the case (See {{AzureRateLimitRetryHandler}} -
> is that wired in correctly, and does it correctly handle 429 response code)?
> There were 390 API calls made in 2mins 22 secs before getting this exception.
> {noformat}
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking resourcegroup:list
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request 300674855: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups?api-version=2015-01-01
> HTTP/1.1
> 2017-01-25 11:57:04,729 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:04,729 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response 300674855: HTTP/1.1 200 OK
> 2017-01-25 11:57:04,732 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking storageaccount:list
> 2017-01-25 11:57:04,733 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request -641560959: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups/aleks-test-res/providers/Microsoft.Storage/storageAccou
> 2017-01-25 11:57:04,763 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,264 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,764 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,938 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:05,938 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response -641560959: HTTP/1.1 429
> 2017-01-25 11:57:05,940 WARN 126 o.a.b.l.j.JcloudsLocation
> [nager-XBhutgvo-4] Error loading available images to report (following
> original error matching template which will be rethrown):
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTT
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:73)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:140)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:109)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
[jira] [Commented] (JCLOUDS-1229) Azure ARM - No retry is the rate limit is exceeded
[
https://issues.apache.org/jira/browse/JCLOUDS-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837816#comment-15837816
]
Aled Sage commented on JCLOUDS-1229:
The problems is that azure-arm does not add the guice module
{{AzureComputeRateLimitModule}} (see {{AzureManagementApiMetadata}}). When
using the jclouds {{ContextBuilder}}, we added it explicitly via
{{.modules(...)}}. With that workaround, the retry works. Without it, there is
no logging about retry and the provisioning fails.
> Azure ARM - No retry is the rate limit is exceeded
> --
>
> Key: JCLOUDS-1229
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1229
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute, jclouds-labs
>Affects Versions: 2.0.0
>Reporter: Thomas Bouron
> Labels: azurecompute-arm
>
> Trying to provision a VM in Azure ARM. My deployment failed because of the
> rate limit exceeded with the stacktrace below.
> I was expecting jclouds to perform a back-off retry as it does for other
> clouds but does not seem to be the case (See {{AzureRateLimitRetryHandler}} -
> is that wired in correctly, and does it correctly handle 429 response code)?
> There were 390 API calls made in 2mins 22 secs before getting this exception.
> {noformat}
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking resourcegroup:list
> 2017-01-25 11:57:04,702 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request 300674855: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups?api-version=2015-01-01
> HTTP/1.1
> 2017-01-25 11:57:04,729 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:04,729 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response 300674855: HTTP/1.1 200 OK
> 2017-01-25 11:57:04,732 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4]
> >> invoking storageaccount:list
> 2017-01-25 11:57:04,733 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Sending request -641560959: GET
> https://management.azure.com/subscriptions/341751b0--45ce-9498-/resourcegroups/aleks-test-res/providers/Microsoft.Storage/storageAccou
> 2017-01-25 11:57:04,763 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,264 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,764 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6]
> Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError:
> service.isUp expected equals true but found false
> 2017-01-25 11:57:05,938 DEBUG 106
> o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4]
> Invalid Expires header (-1); should be in RFC-1123 format; treating as
> already expired: Error parsing data at 0
> 2017-01-25 11:57:05,938 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService
> [nager-XBhutgvo-4] Receiving response -641560959: HTTP/1.1 429
> 2017-01-25 11:57:05,940 WARN 126 o.a.b.l.j.JcloudsLocation
> [nager-XBhutgvo-4] Error loading available images to report (following
> original error matching template which will be rethrown):
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTT
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {x-ms-ratelimit-remaining-subscription-reads=[14947]}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:73)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:140)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:109)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
> at
> org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> at
[jira] [Commented] (JCLOUDS-1108) GCE support for shorten hardwareId
[
https://issues.apache.org/jira/browse/JCLOUDS-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769719#comment-15769719
]
Aled Sage commented on JCLOUDS-1108:
+1 for this. It would be really useful to have for downstream projects such as
Apache Brooklyn. From a user's perspective, it's so much easier to do something
like (brooklyn yaml format below):
{noformat}
location:
google-compute-engine:us-central1-a:
hardwareId: n1-standard-1
{noformat}
Rather than:
{noformat}
location:
google-compute-engine:us-central1-a:
hardwareId:
https://www.googleapis.com/compute/v1/projects/jclouds-gce/zones/us-central1-a/machineTypes/n1-standard-1
{noformat}
Also, see (the closed PR)
https://github.com/jclouds/jclouds-labs-google/pull/171 for some older
discussion of this.
> GCE support for shorten hardwareId
> --
>
> Key: JCLOUDS-1108
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1108
> Project: jclouds
> Issue Type: Improvement
>Reporter: Andrea Turli
>Assignee: Andrea Turli
>
> I'd like to slightly change the MachineTypeToHardware to support hadwareId
> like `f1-micro` instead of the entire URI.
> I've also noticed that location should not be mapped in the HardwareBuilder
> as it is needed only when a the hardware is available only on a particular
> location.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JCLOUDS-1217) Add Canada and London regions to AWS
[
https://issues.apache.org/jira/browse/JCLOUDS-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15754361#comment-15754361
]
Aled Sage commented on JCLOUDS-1217:
When using Apache Brooklyn (which calls down to jclouds), it failed to
provision a VM in {{eu-west-2}} - see
https://issues.apache.org/jira/browse/BROOKLYN-412.
It chose {{m3.medium}} when given {{minRam=1000}}, but I believe that
{{m3.medium}} is not available in that region.
When I supplied an explicit hardwareId and imageId, it provisioned the VM
successfully.
Is it possible to retrieve the set of hardware types via an AWS api, rather
than us hard-coding them in {{AWSEC2HardwareSupplier}}?
> Add Canada and London regions to AWS
>
>
> Key: JCLOUDS-1217
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1217
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore, jclouds-compute
>Affects Versions: 2.0.0
>Reporter: Ignasi Barrera
> Labels: aws-ec2, aws-s3
>
> AWS has added regions in Canada and London. They should be added to the AWS
> compute and blobstore providers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (JCLOUDS-1203) aws-ec2 rate-limiting causes provisioning to fail: need longer back-off/retry
Aled Sage created JCLOUDS-1203:
--
Summary: aws-ec2 rate-limiting causes provisioning to fail: need
longer back-off/retry
Key: JCLOUDS-1203
URL: https://issues.apache.org/jira/browse/JCLOUDS-1203
Project: jclouds
Issue Type: Bug
Affects Versions: 2.0.0, 1.9.2
Reporter: Aled Sage
TL;DR: increase default retry/back-off to 500ms and 6 retries.
In Apache Brooklyn (which uses jclouds), we hit {{Request limit exceeded}} when
provisioning VMs in aws-ec2 [1]. We were provisioning multiple machines
concurrently: different threads were independently calling
{{createNodesInGroup}}. The default exponential backoff and retry within
jclouds wasn't enough.
My understanding is that AWS will rate-limit based on the nature (as well as
number) of API calls. For example, if creating/modifying security groups is a
more expensive operation (from AWS's perspective) than a simple poll for a
machine's state, then those requests would cause rate-limiting sooner.
Within jclouds, the defaults are {{retryCountLimit = 5}} and {{delayStart =
50ms}} (see [2]).
This means we retry with the back-offs being (approximately) 50ms, 100ms,
200ms, 400ms and 500ms.
We overrode the defaults to be 500ms and 6 retries, and could then successfully
provision 20 VMs concurrently. Six of the 20 calls to {{RunInstances}} were
rate-limited. It took several retries before the request was accepted, having
to back off for more than 4 seconds in some cases.
At worst, the existing short back-off may make things worse (the overly
aggressive retry might cause other concurrent calls to also be rate-limited).
At best, the short back-off just isn't long enough so that particular VM
provisioning fails. For example, if AWS uses a leaky bucket algorithm [3] then
hopefully some requests would keep on getting through. But AWS don't publicise
such details of their algorithm/implementation, I believe.
[1] https://issues.apache.org/jira/browse/BROOKLYN-394
[2]
https://github.com/jclouds/jclouds/blob/rel/jclouds-2.0.0/core/src/main/java/org/jclouds/http/handlers/BackoffLimitedRetryHandler.java#L81-#L87
[3] http://en.wikipedia.org/wiki/Leaky_bucket
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JCLOUDS-1202) Provisioning in EC2 failed: IndexOutOfBoundsException when calling addSecurityGroups
[
https://issues.apache.org/jira/browse/JCLOUDS-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681559#comment-15681559
]
Aled Sage commented on JCLOUDS-1202:
[~svet] did I describe it right: that this exception caused provisioning to
fail? (I can't see any try-catch block around it).
[~svet] Do you have the debug log available?
A few lines higher up (in
{{AWSEC2CreateSecurityGroupIfNeeded.createSecurityGroupInRegion}}) it first
creates the security group, then waits for eventual-consistency for the
security group to be visible, and then calls
{{Iterables.get(securityApi.describeSecurityGroupsInRegion(region, name), 0)}}.
It is this last call that throws the {{IndexOutOfBoundsException}}.
That suggests to me one of the following is happening:
1. The eventual consistency guarantees are worse than we thought. For example,
you might query for the security group and see it, but then query again
immediately afterwards and it not be listed.
2. The security group was deleted by some other thread/process in between our
calls.
3. The security group creation failed in some strange way, which meant the
{{securityGroupEventualConsistencyDelay}} check passed but the
{{securityApi.describeSecurityGroupsInRegion}} did not find it.
My guess would be (1) above.
---
One way the code could be fixed is to change what
{{securityGroupEventualConsistencyDelay}} does. It could return the actual
security group, rather than just {{true}} if the security group is eventually
found. This would remove the need to subsequently call
{{securityApi.describeSecurityGroupsInRegion(region, name)}}.
I haven't looked into how {{@Named("SECURITY") Predicate
securityGroupEventualConsistencyDelay}} is defined/used to see if that really
is a good idea.
> Provisioning in EC2 failed: IndexOutOfBoundsException when calling
> addSecurityGroups
>
>
> Key: JCLOUDS-1202
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1202
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 1.9.2
>Reporter: Aled Sage
>
> With jclouds 1.9.2, when provisioning a VM in aws-ec2, it failed with the
> following error:
> {noformat}
> Failed after 1.26s: UncheckedExecutionException:
> java.lang.IndexOutOfBoundsException: position (0) must be less than the
> number of elements that remained (0)
> com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.IndexOutOfBoundsException: position (0) must be less than the
> number of elements that remained (0)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4827)
> at
> org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.getSecurityGroupsForTagAndOptions(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:183)
> at
> org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.addSecurityGroups(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:109)
> at
> org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.addSecurityGroups(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:185)
> at
> org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:80)
> at
> org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:79)
> at
> org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:50)
> at
> org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.createKeyPairAndSecurityGroupsAsNeededThenRunInstances(EC2CreateNodesInGroupThenAddToSet.java:213)
> at
> org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.runInstancesAndWarnOnInvisible(EC2CreateNodesInGroupThenAddToSet.java:151)
> at
> org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.execute(EC2CreateNodesInGroupThenAddToSet.java:132)
> at
> org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:210)
> at
>
[jira] [Created] (JCLOUDS-1202) Provisioning in EC2 failed: IndexOutOfBoundsException when calling addSecurityGroups
Aled Sage created JCLOUDS-1202:
--
Summary: Provisioning in EC2 failed: IndexOutOfBoundsException
when calling addSecurityGroups
Key: JCLOUDS-1202
URL: https://issues.apache.org/jira/browse/JCLOUDS-1202
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 1.9.2
Reporter: Aled Sage
With jclouds 1.9.2, when provisioning a VM in aws-ec2, it failed with the
following error:
{noformat}
Failed after 1.26s: UncheckedExecutionException:
java.lang.IndexOutOfBoundsException: position (0) must be less than the number
of elements that remained (0)
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.IndexOutOfBoundsException: position (0) must be less than the number
of elements that remained (0)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
at
com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4827)
at
org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.getSecurityGroupsForTagAndOptions(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:183)
at
org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.addSecurityGroups(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:109)
at
org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.addSecurityGroups(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:185)
at
org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:80)
at
org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:79)
at
org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:50)
at
org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.createKeyPairAndSecurityGroupsAsNeededThenRunInstances(EC2CreateNodesInGroupThenAddToSet.java:213)
at
org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.runInstancesAndWarnOnInvisible(EC2CreateNodesInGroupThenAddToSet.java:151)
at
org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.execute(EC2CreateNodesInGroupThenAddToSet.java:132)
at
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:210)
at
org.jclouds.ec2.compute.EC2ComputeService.createNodesInGroup(EC2ComputeService.java:149)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:735)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:626)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:459)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:449)
at
org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:106)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:431)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:407)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IndexOutOfBoundsException: position (0) must be less than
the number of elements that remained (0)
at com.google.common.collect.Iterators.get(Iterators.java:813)
at com.google.common.collect.Iterables.get(Iterables.java:724)
at
org.jclouds.aws.ec2.compute.loaders.AWSEC2CreateSecurityGroupIfNeeded.createSecurityGroupInRegion(AWSEC2CreateSecurityGroupIfNeeded.java:107)
at
[jira] [Created] (JCLOUDS-1187) Avoid excessive memory usage when processing massive http response message
Aled Sage created JCLOUDS-1187:
--
Summary: Avoid excessive memory usage when processing massive http
response message
Key: JCLOUDS-1187
URL: https://issues.apache.org/jira/browse/JCLOUDS-1187
Project: jclouds
Issue Type: Improvement
Reporter: Aled Sage
With jclouds 1.9.3-SNAPSHOT and 2.0.0-SNAPSHOT (and all GA versions)...
If jclouds receives a crazily big http response (e.g. see
https://github.com/jclouds/jclouds/pull/1020), then jclouds consumes a huge
amount of memory while processing that response. It holds multiple copies of
the response in-memory at the same time.
As reported in https://issues.apache.org/jira/browse/BROOKLYN-364, the memory
usage (according to {{jmap -histo:live }}) is most char arrays (which is
what backs the StringBuilder and String). When processing a 154MB response
payload, the memory used by char arrays goes something like this: 66MB -> 104MB
-> 323MB -> 625MB -> 939MB -> 461MB -> 633MB -> 327MB -> 21MB.
(I don't know how much to believe the {{:live}}: is that definitely the size of
char arrays that could not be GC'ed?)
There are two main areas where this memory is consumed.
*Wire Log*
In the jclouds wire log, the following happens (approximately?!):
* We read the response input stream, writing it to a {{FileBackedOutputStream}}
- nice!
* In {{Wire.wire(String, InputStream)}}, we read the {{FileBackedOutputStream}}
into a {{StringBuilder}}, and then call
{{getWireLog().debug(buffer.toString())}}:
The StringBuilder holds the {{char[]}}; the toString() duplicates it - so two
copies in memory.
Unfortunately in the Softlayer example, it's all one huge line, so we logged
it all in one go.
I think (but need to dig into it more) that the logging framework (slf4j ->
log4j in my case) ends up with multiple copies as well, while processing the
call to {{log.debug(String)}}. (Hence peaking at 939MB of char arrays in
memory).
When the method returns, all these copies can be GC'ed.
* The response payload has now been switched to the {{FileBackedOutputStream}},
so that will be used subsequently.
*Json Parsing*
To parse the HTTP response:
* The response is passed to
{{org.jclouds.http.functions.ParseJson.apply(HttpResponse)}}
* This calls {{json.fromJson(Strings2.toStringAndClose(stream), type)}}.
* The {{Strings2.toStringAndClose}} calls {{CharStreams.toString(new
InputStreamReader(input, Charsets.UTF_8))}}.
This reads the stream into a StringBuilder, then calls toString - so holds
two copies in memory.
This explains the second spike in memory usage (though I'm surprised it went
as high as 633MB of char arrays in memory).
When the method returns, we have our one String.
*Possible Improvements to Wire Log*
{{org.jclouds.logging.internal.Wire}} could be configurable to only log the
first X bytes of a response (so crazily long messages would be truncated in the
log).
Alternatively/additionally, {{Wire.wire}} could force splitting a huge line
into multiple log messages when calling
{{getWireLog().debug(buffer.toString())}}. Again this could be configurable.
In production usage, I personally would always configure it to truncate: better
to miss the end of the response rather than risk an {{OutOfMemoryError}}. Note
this particular part isn't an issue if jclouds.wire is set to INFO or higher.
*Possible Improvements to Json Parsing*
I think {{ParseJson.apply(InputStream stream, Type type)}} should pass the
{{new InputStreamReader(inputStream)}} to {{org.jclouds.json.Json.fromJson()}}
(in a try-finally to close the stream, obviously).
This would require adding a new method to the interface
{{org.jclouds.json.Json}}.
The implementation of {{GsonWrapper}} can handle that easily: it can call
{{gson.fromJson(Reader, Type)}} instead of {{gson.fromJson(String, Type)}}. It
looks to me like that will parse the stream as it reads it, rather than having
to hold the whole string in memory at once.
If we do these improvements, I don't think we'll ever hold the full 154MB char
array in memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (JCLOUDS-1178) AWS vm deletion failed: did not retry on response "The service is unavailable. Please try again shortly"
Aled Sage created JCLOUDS-1178:
--
Summary: AWS vm deletion failed: did not retry on response "The
service is unavailable. Please try again shortly"
Key: JCLOUDS-1178
URL: https://issues.apache.org/jira/browse/JCLOUDS-1178
Project: jclouds
Issue Type: Bug
Affects Versions: 1.9.2
Reporter: Aled Sage
While using jclouds 1.9.2, while attempting to stop a VM in
{{aws-ec2:us-east-1}} I hit the error below:
{noformat}
Failed after 39.1s: Error invoking stop at MySqlNodeImpl{id=zrxtil48g6}:
AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/
HTTP/1.1 failed with code 503, error:
AWSError{requestId='d4a38d1b-4f02-4161-b7ff-6cd78bdefe41', requestToken='null',
code='Unavailable', message='The service is unavailable. Please try again
shortly.', context='{Response=, Errors=}'}
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException:
Error invoking stop at MySqlNodeImpl{id=zrxtil48g6}: AWSResponseException:
request POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code
503, error: AWSError{requestId='d4a38d1b-4f02-4161-b7ff-6cd78bdefe41',
requestToken='null', code='Unavailable', message='The service is unavailable.
Please try again shortly.', context='{Response=, Errors=}'}
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException.propagate(EffectorUtils.java:318)
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException.access$100(EffectorUtils.java:285)
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils.handleEffectorException(EffectorUtils.java:326)
at
org.apache.brooklyn.core.effector.EffectorTasks$EffectorBodyTaskFactory$2.handleException(EffectorTasks.java:90)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask.handleException(DynamicSequentialTask.java:469)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:417)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/
HTTP/1.1 failed with code 503, error:
AWSError{requestId='d4a38d1b-4f02-4161-b7ff-6cd78bdefe41', requestToken='null',
code='Unavailable', message='The service is unavailable. Please try again
shortly.', context='{Response=, Errors=}'}
at
org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:129)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.doStop(MachineLifecycleEffectorTasks.java:863)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.stop(MachineLifecycleEffectorTasks.java:743)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$StopEffectorBody.call(MachineLifecycleEffectorTasks.java:308)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$StopEffectorBody.call(MachineLifecycleEffectorTasks.java:305)
at
org.apache.brooklyn.core.effector.EffectorTasks$EffectorBodyTaskFactory$1.call(EffectorTasks.java:82)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)
... 5 more
Caused by: java.util.concurrent.ExecutionException:
org.jclouds.aws.AWSResponseException: request POST
https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 503, error:
AWSError{requestId='d4a38d1b-4f02-4161-b7ff-6cd78bdefe41', requestToken='null',
code='Unavailable', message='The service is unavailable. Please try again
shortly.', context='{Response=, Errors=}'}
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
at org.apache.brooklyn.util.core.task.BasicTask.get(BasicTask.java:361)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.doStop(MachineLifecycleEffectorTasks.java:851)
... 10 more
Caused by: org.jclouds.aws.AWSResponseException: request POST
https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 503, error:
AWSError{requestId='d4a38d1b-4f02-4161-b7ff-6cd78bdefe41', requestToken='null',
code='Unavailable', message='The service is unavailable.
[jira] [Created] (JCLOUDS-1165) wrong loginUser inferred: add support for trying more possible loginUser values
Aled Sage created JCLOUDS-1165:
--
Summary: wrong loginUser inferred: add support for trying more
possible loginUser values
Key: JCLOUDS-1165
URL: https://issues.apache.org/jira/browse/JCLOUDS-1165
Project: jclouds
Issue Type: Improvement
Affects Versions: 1.9.2
Reporter: Aled Sage
Sometimes when provisioning VMs, provisioning fails because ssh'ing to the VM
fails - it is the wrong {{loginUser}}.
The workaround is simple: to figure out the right loginUser for the VM image
that is being chosen, and to call {{templateOptions.overrideLoginUser()}}.
For situations where the default loginUser is wrong, I'd like alternative ones
to also be tried.
One way would be to add {{TemplateOptions.fallbackLoginUsers(Iterable
fallbackLoginUsers)}}. When waiting for the first ssh connection, it could try
these fallbacks as well. The first user to work with the given credentials
could then be used subsequently.
If this option were available, I'd probably always pass in
{{templateOptions.fallbackLoginUsers(ImmutableList.of("root", "centos",
"ubuntu", "ec2-user"))}}.
Another option would be that if there is no explicit (i.e. no overridden) login
user in the templateOptions, then we try a bunch of stock users. We could
adjust the existing {{PopulateDefaultLoginCredentialsForImageStrategy}} to
return a set of {{LoginCredentials}} to try - but that would be a bit harder to
change while preserving backwards compatibility.
---
The existing logic to figure out the default loginUser looks pretty scary - for
example, [1, 2, 3]. There are hard-coded rules for whether to use "ubuntu",
"ec2-user", etc based on the OS type, the cloud provider, and the image owner!
I had previously thought that image metadata from the cloud provider would tell
us the right loginUser, but I guess that's not the case!
The workaround is to explicitly set the loginUser in the location definition.
[1]
https://github.com/jclouds/jclouds/blob/rel/jclouds-1.9.2/apis/ec2/src/main/java/org/jclouds/ec2/compute/strategy/EC2PopulateDefaultLoginCredentialsForImageStrategy.java#L63-L67
[2]
https://github.com/jclouds/jclouds/blob/rel/jclouds-1.9.2/compute/src/main/java/org/jclouds/compute/strategy/impl/ReturnCredentialsBoundToImage.java
[3]
https://github.com/jclouds/jclouds/blob/rel/jclouds-1.9.2/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/compute/config/NovaComputeServiceContextModule.java#L190-L193
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (JCLOUDS-1100) OpenStack provisioning fails: exception "cidrBlock ::/0 is not a valid CIDR"
Aled Sage created JCLOUDS-1100:
--
Summary: OpenStack provisioning fails: exception "cidrBlock ::/0
is not a valid CIDR"
Key: JCLOUDS-1100
URL: https://issues.apache.org/jira/browse/JCLOUDS-1100
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 1.9.2
Reporter: Aled Sage
When provisioning a VM to OpenStack (in IBM's BlueBox), I got the exception
below.
{noformat}
Failed after 6.11s: Error invoking start at DockerHostImpl{id=uMtN8h3W}:
UncheckedExecutionException: java.lang.IllegalArgumentException: cidrBlock ::/0
is not a valid CIDR
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException:
Error invoking start at DockerHostImpl{id=uMtN8h3W}:
UncheckedExecutionException: java.lang.IllegalArgumentException: cidrBlock ::/0
is not a valid CIDR
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException.propagate(EffectorUtils.java:299)
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException.access$100(EffectorUtils.java:266)
at
org.apache.brooklyn.core.mgmt.internal.EffectorUtils.handleEffectorException(EffectorUtils.java:306)
at
org.apache.brooklyn.core.effector.EffectorTasks$EffectorBodyTaskFactory$2.handleException(EffectorTasks.java:90)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask.handleException(DynamicSequentialTask.java:469)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:417)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:518)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
UncheckedExecutionException: java.lang.IllegalArgumentException: cidrBlock ::/0
is not a valid CIDR
at
org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:128)
at
org.apache.brooklyn.util.core.task.BasicTask.getUnchecked(BasicTask.java:372)
at org.apache.brooklyn.util.core.task.Tasks$2.get(Tasks.java:285)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.preStartAtMachineAsync(MachineLifecycleEffectorTasks.java:412)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.startInLocation(MachineLifecycleEffectorTasks.java:339)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.startInLocations(MachineLifecycleEffectorTasks.java:324)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks.start(MachineLifecycleEffectorTasks.java:313)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$StartEffectorBody.call(MachineLifecycleEffectorTasks.java:214)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$StartEffectorBody.call(MachineLifecycleEffectorTasks.java:201)
at
org.apache.brooklyn.core.effector.EffectorTasks$EffectorBodyTaskFactory$1.call(EffectorTasks.java:82)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)
... 5 more
Caused by: java.util.concurrent.ExecutionException:
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.IllegalArgumentException: cidrBlock ::/0 is not a valid CIDR
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
at org.apache.brooklyn.util.core.task.BasicTask.get(BasicTask.java:361)
at
org.apache.brooklyn.util.core.task.BasicTask.getUnchecked(BasicTask.java:370)
... 14 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.IllegalArgumentException: cidrBlock ::/0 is not a valid CIDR
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
at
com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4736)
at
org.apache.brooklyn.location.jclouds.networking.JcloudsLocationSecurityGroupCustomizer.setSecurityGroupOnTemplate(JcloudsLocationSecurityGroupCustomizer.java:389)
at
[jira] [Created] (JCLOUDS-1097) NullPointerException in azurecompute provisioning, when logging request failure
Aled Sage created JCLOUDS-1097:
--
Summary: NullPointerException in azurecompute provisioning, when
logging request failure
Key: JCLOUDS-1097
URL: https://issues.apache.org/jira/browse/JCLOUDS-1097
Project: jclouds
Issue Type: Bug
Components: jclouds-labs
Affects Versions: 1.9.2
Reporter: Aled Sage
When provisioning a VM in azure, using the jclouds-labs azurecompute 1.9.2
release, I hit the NullPointerException shown below. The jclouds
HttpResponseException marks {{getResponse}} as {{@Nullable}}, so we should
guard against null when doing the logging in azurecompute.
{noformat}
1) ExecutionException on brookly-o4gv3w-amp-auto-mgid-riak-xaoo-e7e:
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at
com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at
org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:123)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.jclouds.azurecompute.util.ConflictManagementPredicate.apply(ConflictManagementPredicate.java:157)
at
org.jclouds.azurecompute.compute.AzureComputeServiceAdapter.createNodeWithGroupEncodedIntoName(AzureComputeServiceAdapter.java:147)
at
org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.createNodeWithGroupEncodedIntoName(AdaptingComputeServiceStrategies.java:195)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet$AddNode.call(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:79)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet$AddNode.call(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:63)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at java.lang.Thread.getStackTrace(Thread.java:1589)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.createNodeInGroupWithNameAndTemplate(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:167)
at
org.jclouds.azurecompute.compute.strategy.GetOrCreateStorageServiceAndVirtualNetworkThenCreateNodes.createNodeInGroupWithNameAndTemplate(GetOrCreateStorageServiceAndVirtualNetworkThenCreateNodes.java:89)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:122)
at
org.jclouds.azurecompute.compute.strategy.GetOrCreateStorageServiceAndVirtualNetworkThenCreateNodes.execute(GetOrCreateStorageServiceAndVirtualNetworkThenCreateNodes.java:122)
at
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:210)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:723)
at
org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:613)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:406)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:396)
at
org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:98)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:380)
at
org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:364)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:518)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
... 3 more
{noformat}
FYI the request that seemed to fail was {{POST
https://management.core.windows.net/----/services/hostedservices/brookly-o4gv3w-amp-auto-mgid-riak-xaoo-e7e/deployments
HTTP/1.1}}. It then logged {{Command not considered safe to retry
[jira] [Commented] (JCLOUDS-889) listImages don't consider `hvm` vs `paravirtual`
[
https://issues.apache.org/jira/browse/JCLOUDS-889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498313#comment-14498313
]
Aled Sage commented on JCLOUDS-889:
---
For example, on aws-ec2 with minRam=8192, one of the images offered to the
`TempateBuilder.imageChooser` was eu-west-1/ami-7db2d50a (a Canonical-published
image for Ubuntu 14.04). The instance type inferred by jclouds was r3.large.
However, this gave the error:
org.jclouds.aws.AWSResponseException: request POST
https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='c72a6249-31b7-44c4-a30b-013527ebe249',
requestToken='null', code='InvalidParameterCombination',
message='Virtualization type 'hvm' is required for instances of type
'r3.large'.', context='{Response=, Errors=}'}
listImages don't consider `hvm` vs `paravirtual`
Key: JCLOUDS-889
URL: https://issues.apache.org/jira/browse/JCLOUDS-889
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 1.9.0
Reporter: Andrea Turli
the node creation fails if one only specifies `osFamily` and `minRam=8G`
(which triggers the creation of a large instance type) as `hvm` is an
implicit requirement not satisfied by the templateBuilder
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JCLOUDS-207) Key Pair and Security Groups created by jclouds are not removed when the node is destroyed (via Jclouds)
[ https://issues.apache.org/jira/browse/JCLOUDS-207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263538#comment-14263538 ] Aled Sage commented on JCLOUDS-207: --- There seems to be eventual consistency within EC2 for detecting when the security group is no longer in use. After terminating the VM, it can take anywhere from immediate to several minutes before an attempted deletion of the security group will succeed (it fails if it thinks there is still a VM using the security group). jclouds does a retry 3 times to delete it, but it doesn't wait for several minutes. An alternative could be: on delete, query EC2 to find the VMs using the security group, and then query those VMs' states to see if they are not termianted (or short-cut that if we know we currently deleting the only VM in that list) - we could then keep trying for 5 or 10 minutes. But that doesn't sound ideal either. Key Pair and Security Groups created by jclouds are not removed when the node is destroyed (via Jclouds) Key: JCLOUDS-207 URL: https://issues.apache.org/jira/browse/JCLOUDS-207 Project: jclouds Issue Type: Bug Affects Versions: 1.6.0 Reporter: Eugen Paraschiv Labels: ec2 Simply stated, the problem is that the nodes that are created in EC2 via jclouds leave a key-pair and a security group each, after they're deleted (also via jclouds). This issue is described in much more detail here: http://www.cloudsoftcorp.com/blog/tidying-up-after-jclouds/ Hopefully it's an easy fix and the operation of first creating and then destroying the node will leave no unnecessary artifacts on the EC2 account. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (JCLOUDS-529) Getting exception while destroying an aws instance using jclouds
[
https://issues.apache.org/jira/browse/JCLOUDS-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260444#comment-14260444
]
Aled Sage commented on JCLOUDS-529:
---
See fix in https://github.com/jclouds/jclouds/pull/629 (against 1.8.x branch
currently).
Previously, the error code was always `InvalidGroup.InUse`. Now some people are
getting `DependencyVioloation`.
In
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DeleteSecurityGroup.html
it says: the operation fails with InvalidGroup.InUse in EC2-Classic or
DependencyViolation in EC2-VPC. I wonder if a default has been changed for new
accounts versus old accounts perhaps?
The fix is simple: to interpret the `DependencyVioloation` in the same was as
the `InUse`.
Getting exception while destroying an aws instance using jclouds
Key: JCLOUDS-529
URL: https://issues.apache.org/jira/browse/JCLOUDS-529
Project: jclouds
Issue Type: Bug
Components: jclouds-compute, jclouds-core
Reporter: supraja
Getting the below Exception while trying to destroy an aws instance using
destroyNode(id)
org.jclouds.aws.AWSResponseException: request POST
https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='2de
fa53c-fd1f-4cad-b547-4ffd2158e020', requestToken='null',
code='DependencyViolation', message='resource sg-ba812bdf has a dependent
object', context='{Response=,
Errors=}'}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (JCLOUDS-722) Gaps in CloudStack 4.x API coverage
Aled Sage created JCLOUDS-722: - Summary: Gaps in CloudStack 4.x API coverage Key: JCLOUDS-722 URL: https://issues.apache.org/jira/browse/JCLOUDS-722 Project: jclouds Issue Type: Improvement Affects Versions: 1.8.0 Reporter: Aled Sage Priority: Minor There are many gaps in the jclouds coverage of the latest CloudStack 4.x API [1]. The list below is *not* exhaustive. Missing APIs include: * the entire API for: ** VPN ** AutoScale ** VPC ** Network ACL ** Affinity group ** User ** Resource tags ** Nic ** Resource metadata ** Cloud identifier ** API Discovery * parts of the API for: ** org.jclouds.cloudstack.features.LoadBalancerApi, such as createLBStickinessPolicy, uploadSslCert, etc ** org.jclouds.cloudstack.features.VirtualMachineApi, such as restoreVirtualMachine, scaleVirtualMachine, addNicToVirtualMachine [1] http://cloudstack.apache.org/docs/api/apidocs-4.4/TOC_User.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (JCLOUDS-723) CloudStack createNodesInGroup fails for service providers with locked down APIs
Aled Sage created JCLOUDS-723: - Summary: CloudStack createNodesInGroup fails for service providers with locked down APIs Key: JCLOUDS-723 URL: https://issues.apache.org/jira/browse/JCLOUDS-723 Project: jclouds Issue Type: Bug Affects Versions: 1.8.0 Reporter: Aled Sage Creating VM(s) on CloudStack fails with some service providers, because they lock down access to parts of their API. For example, API calls made by the listImages method are sometimes forbidden. CloudStackComputeServiceAdapter.listImages: https://github.com/apache/jclouds/blob/f17c876d8dc161988f586c3cf343361d896f6928/apis/cloudstack/src/main/java/org/jclouds/cloudstack/compute/strategy/CloudStackComputeServiceAdapter.java#L284-294 The method tries to list all templates. First, it lists all templates that are executable. Then, it lists all templates associated with each project in the account. Translated to Cloudmonkey-suitable commands, the call flow is: * list templates listAll=true templatefilter=executable * list accounts listAll=true for each account response: extract name and domainid from response and call: * list projects listAll=true account=.. domainid=.. jclouds fails because it gets a response 405 Method Not Allowed to the listAccounts call (and would do the same for the listProjects call if it got that far). /api/CloudPlatformProxy?apiKey=removedcommand=listAccountsexpires=2014-07-21T11%3A08%3A32%2Bresponse=jsonsignatureversion=3signature=removed HTTP/1.1 405 Method Not Allowed Cache-Control: no-cache Pragma: no-cache Expires: -1 Server: Microsoft-IIS/7.5 X-AspNet-Version: 4.0.30319 X-Powered-By: ASP.NET Date: Mon, 21 Jul 2014 11:00:19 GMT Content-Length: 0 /api/CloudPlatformProxy?apiKey=removedcommand=listProjectsexpires=2014-07-21T11%3A04%3A59%2Bresponse=jsonsignatureversion=3signature=removed HTTP/1.1 405 Method Not Allowed Cache-Control: no-cache Pragma: no-cache Expires: -1 Server: Microsoft-IIS/7.5 X-AspNet-Version: 4.0.30319 X-Powered-By: ASP.NET Date: Mon, 21 Jul 2014 10:57:13 GMT Content-Length: 0 The cloud provider's response was: Accounts and projects are blocked simply because this is a multi tenant service where account isolation is important. So we don’t allow users to list all accounts on the platform as each one is tied to a customer. Projects and domain aren’t exposed because we haven’t assessed the risks to billing if these are enabled. The credentials that we give you will tie you to an account and then (other than domains and projects) you can do what you want. I like the idea of enabling certain list API calls but only when listall is set to false. Of course if its just stopping a test program then the incentive in fixing it would be minimal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (JCLOUDS-724) CloudStack live tests: skip instead of fail if feature not available
Aled Sage created JCLOUDS-724: - Summary: CloudStack live tests: skip instead of fail if feature not available Key: JCLOUDS-724 URL: https://issues.apache.org/jira/browse/JCLOUDS-724 Project: jclouds Issue Type: Improvement Affects Versions: 1.8.0 Reporter: Aled Sage Priority: Minor When running the CloudStack live test suite against a cloud provider, I got a lot of failures that are purely because parts of the API are not exposed by the cloud provider. For example, all ISO tests failed. It would be nice to skip tests rather than fail if the operations are not supported. The test could first do listApis [1]. If that feature is not available for the account being used, then it could throw an org.testng.SkipException. [1] http://cloudstack.apache.org/docs/api/apidocs-4.4/user/listApis.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (JCLOUDS-725) CloudStack guide missing
Aled Sage created JCLOUDS-725: - Summary: CloudStack guide missing Key: JCLOUDS-725 URL: https://issues.apache.org/jira/browse/JCLOUDS-725 Project: jclouds Issue Type: Improvement Reporter: Aled Sage Priority: Minor There is no guide for getting started with CloudStack at https://jclouds.apache.org/guides/ It would be great to have a new guide that follows a similar structure to those for other APIs, such as http://jclouds.apache.org/guides/openstack/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (JCLOUDS-637) CloudStack uses generateSecurityGroup=false, i.e. uses default security group
Aled Sage created JCLOUDS-637: - Summary: CloudStack uses generateSecurityGroup=false, i.e. uses default security group Key: JCLOUDS-637 URL: https://issues.apache.org/jira/browse/JCLOUDS-637 Project: jclouds Issue Type: Bug Components: jclouds-compute Affects Versions: 1.7.3 Reporter: Aled Sage When creating a VM, the CloudStack integration by default uses the default security group. It does not behave like other clouds to create a new security group with the correct opened ports. There is a configuration option `CloudStackTemplateOptions.generateSecurityGroup`. https://issues.apache.org/jira/browse/JCLOUDS-106 looks related, but is marked resolved. I'd have expected it to behave like for other cloud providers, to give a consistent cloud abstraction. This would involve creating a new security group. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (JCLOUDS-638) CloudStackTemplateOptions.securityGroup not supported correctly
Aled Sage created JCLOUDS-638: - Summary: CloudStackTemplateOptions.securityGroup not supported correctly Key: JCLOUDS-638 URL: https://issues.apache.org/jira/browse/JCLOUDS-638 Project: jclouds Issue Type: Bug Components: jclouds-compute Affects Versions: 1.7.3 Reporter: Aled Sage The CloudStack integration uses the security group id (see CloudStackTemplateOptions.securityGroupId). The generic feature of TemplateOptions.securityGroup (which expects a name) is not used in CloudStackComputeServiceAdapter. I'd have expected CloudStack integration to work the same way as other cloud providers through jclouds - i.e. to be able to pass the security group names. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (JCLOUDS-557) jclouds fails on CloudStack 4.3 - listProjects with domainid fails
[ https://issues.apache.org/jira/browse/JCLOUDS-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066789#comment-14066789 ] Aled Sage commented on JCLOUDS-557: --- https://issues.apache.org/jira/browse/CLOUDSTACK-6508 is now fixed in CloudStack master: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=d382be9 jclouds fails on CloudStack 4.3 - listProjects with domainid fails -- Key: JCLOUDS-557 URL: https://issues.apache.org/jira/browse/JCLOUDS-557 Project: jclouds Issue Type: Bug Components: jclouds-compute Affects Versions: 1.7.2 Reporter: Aled Sage I tried to use jclouds to create a VM in a CloudPlatform 4.3 environment (i.e. CloudStack 4.3). It failed with the exception below. The problem I hit was https://issues.apache.org/jira/browse/CLOUDSTACK-6508. As described in that bug report, com.cloud.api.query.QueryManagerImpl.listProjectsInternal has a bug where it will fail when domainId is supplied. I was able to work around this by patching the jclouds code org.jclouds.cloudstack.suppliers.ProjectsForCurrentUser so that it passed in just the account and not the domainid. However, that does *not* feel like a solution that should be committed! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (JCLOUDS-622) Guava 18 compatibility
[ https://issues.apache.org/jira/browse/JCLOUDS-622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065517#comment-14065517 ] Aled Sage commented on JCLOUDS-622: --- Is deleting `Strings2.toString(InputSupplier)` rather than deprecating it too aggressive? Though guava 18 will delete methods using `InputSupplier`, the interface will stick around until either June or December 2015 [1,2]. So we can deprecate toString(InputSupplier) in 1.8 and delete it in jclouds 1.9, as per the jclouds deprecation policy? See https://github.com/jclouds/jclouds-labs/pull/69 for downstream fixes to remove usage of this method. [1] https://code.google.com/p/guava-libraries/wiki/Release16 [2] http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/InputSupplier.html Guava 18 compatibility -- Key: JCLOUDS-622 URL: https://issues.apache.org/jira/browse/JCLOUDS-622 Project: jclouds Issue Type: Bug Affects Versions: 1.8.0 Reporter: Andrew Gaul Assignee: Andrew Gaul Guava 16 deprecated and Guava 18 will remove many InputSupplier methods that jclouds relies on. jclouds should use alternate methods to allow users to upgrade to Guava 18 when Google releases it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (JCLOUDS-557) jclouds fails on CloudStack 4.3 - listProjects with domainid fails
Aled Sage created JCLOUDS-557: - Summary: jclouds fails on CloudStack 4.3 - listProjects with domainid fails Key: JCLOUDS-557 URL: https://issues.apache.org/jira/browse/JCLOUDS-557 Project: jclouds Issue Type: Bug Components: jclouds-compute Affects Versions: 1.7.2 Reporter: Aled Sage I tried to use jclouds to create a VM in a CloudPlatform 4.3 environment (i.e. CloudStack 4.3). It failed with the exception below. The problem I hit was https://issues.apache.org/jira/browse/CLOUDSTACK-6508. As described in that bug report, com.cloud.api.query.QueryManagerImpl.listProjectsInternal has a bug where it will fail when domainId is supplied. I was able to work around this by patching the jclouds code org.jclouds.cloudstack.suppliers.ProjectsForCurrentUser so that it passed in just the account and not the domainid. However, that does *not* feel like a solution that should be committed! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (JCLOUDS-549) NullPointerException in LoginCredentials.toString
Aled Sage created JCLOUDS-549: - Summary: NullPointerException in LoginCredentials.toString Key: JCLOUDS-549 URL: https://issues.apache.org/jira/browse/JCLOUDS-549 Project: jclouds Issue Type: Bug Components: jclouds-core Affects Versions: 1.7.2 Reporter: Aled Sage Priority: Minor Fix For: 1.7.3 If a LoginCredentials is built with a null password or null privateKey then toString will throw an NPE. See fix in https://github.com/jclouds/jclouds/pull/353 -- This message was sent by Atlassian JIRA (v6.2#6252)
