[jira] [Commented] (JCLOUDS-1527) Azure compute rate limit exceeded exception during the build of image cache
[
https://issues.apache.org/jira/browse/JCLOUDS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979989#comment-16979989
]
Simone commented on JCLOUDS-1527:
-
I think that your solution is a good optimization (and the best with less
effort), but in these days I have not time to implement it :(
> Azure compute rate limit exceeded exception during the build of image cache
> ---
>
> Key: JCLOUDS-1527
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1527
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 2.1.3
>Reporter: Simone
>Priority: Critical
> Labels: azurecompute-arm
>
> Here the exception:
> {noformat}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:79)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:138)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:107)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
> at
> org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> at
> com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:84)
> at com.sun.proxy.$Proxy267.getVersion
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.getImagesFromPublisher(AzureComputeServiceAdapter.java:212)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImagesByLocation(AzureComputeServiceAdapter.java:226)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImages(AzureComputeServiceAdapter.java:255)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:121)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:118)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:75)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:57)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
> at com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:150)
> at
> org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:106)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:853)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:665)
> {noformat}
> I use the Azure passing the imageId to template builder, I think that the
> cache is usefull in my case, but passing the override parameter:
> "jclouds.azurecompute.arm.publishers" to an empty string breaks line: 666
> (:p) in TemplateBuilderImpl
> "checkState(!images.isEmpty(), "no images present!");"
> Can I make a PR to change that line, to explode when images are empty but
> only when no image id is supplied? Do you think it could be safe?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (JCLOUDS-1527) Azure compute rate limit exceeded exception during the build of image cache
[
https://issues.apache.org/jira/browse/JCLOUDS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978190#comment-16978190
]
Ignasi Barrera commented on JCLOUDS-1527:
-
Looking at those loops, I think it would make sense to restrict the versions
just to {{latest}} by default, and have a context property to enable listing
all versions for each SKU. The Operating system version (such as 7, for CentOS)
is usually stored in the SKU, and there might be many versions for that OS
version. I think a reasonable approach is to just list {{latest}} by default,
and that would remove N iterations for each SKU, which should considerably
reduce the network calls.
Would that work?
> Azure compute rate limit exceeded exception during the build of image cache
> ---
>
> Key: JCLOUDS-1527
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1527
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 2.1.3
>Reporter: Simone
>Priority: Critical
> Labels: azurecompute-arm
>
> Here the exception:
> {noformat}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:79)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:138)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:107)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
> at
> org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> at
> com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:84)
> at com.sun.proxy.$Proxy267.getVersion
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.getImagesFromPublisher(AzureComputeServiceAdapter.java:212)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImagesByLocation(AzureComputeServiceAdapter.java:226)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImages(AzureComputeServiceAdapter.java:255)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:121)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:118)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:75)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:57)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
> at com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:150)
> at
> org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:106)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:853)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:665)
> {noformat}
> I use the Azure passing the imageId to template builder, I think that the
> cache is usefull in my case, but passing the override parameter:
> "jclouds.azurecompute.arm.publishers" to an empty string breaks line: 666
> (:p) in TemplateBuilderImpl
> "checkState(!images.isEmpty(), "no images present!");"
> Can I make a PR to change that line, to explode when images are empty but
> only when no
[jira] [Commented] (JCLOUDS-1527) Azure compute rate limit exceeded exception during the build of image cache
[
https://issues.apache.org/jira/browse/JCLOUDS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976476#comment-16976476
]
Simone commented on JCLOUDS-1527:
-
I am sorry but I have already adjusted the publisher list, so the last chance
to fix it is to refactor the fors at
org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.getImagesFromPublisher(AzureComputeServiceAdapter.java:212)
but it doesn't look simple to me :D
> Azure compute rate limit exceeded exception during the build of image cache
> ---
>
> Key: JCLOUDS-1527
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1527
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 2.1.3
>Reporter: Simone
>Priority: Critical
> Labels: azurecompute-arm
>
> Here the exception:
> {noformat}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:79)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:138)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:107)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
> at
> org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> at
> com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:84)
> at com.sun.proxy.$Proxy267.getVersion
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.getImagesFromPublisher(AzureComputeServiceAdapter.java:212)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImagesByLocation(AzureComputeServiceAdapter.java:226)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImages(AzureComputeServiceAdapter.java:255)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:121)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:118)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:75)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:57)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
> at com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:150)
> at
> org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:106)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:853)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:665)
> {noformat}
> I use the Azure passing the imageId to template builder, I think that the
> cache is usefull in my case, but passing the override parameter:
> "jclouds.azurecompute.arm.publishers" to an empty string breaks line: 666
> (:p) in TemplateBuilderImpl
> "checkState(!images.isEmpty(), "no images present!");"
> Can I make a PR to change that line, to explode when images are empty but
> only when no image id is supplied? Do you think it could be safe?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (JCLOUDS-1527) Azure compute rate limit exceeded exception during the build of image cache
[
https://issues.apache.org/jira/browse/JCLOUDS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976420#comment-16976420
]
Ignasi Barrera commented on JCLOUDS-1527:
-
Changing this is not that trivial. The image cache list is not only used when
provisioning VMs, but also in many other places, such as when listing images,
or even when listing existing VMs (there is a process that tries to get the
image for each node that uses the image cache). If the list of images is empty,
many things will break, so I don't think changing that line is the right fix.
In the end, the TemplateBuilder semantics is just about the selection of images
from the existing image set, so there is no point in making it work when there
are no images there.
The right fix should be properly adjusting the publisher list to the one that
contains the image(s) you are using. Can you configure that instead?
> Azure compute rate limit exceeded exception during the build of image cache
> ---
>
> Key: JCLOUDS-1527
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1527
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-compute
>Affects Versions: 2.1.3
>Reporter: Simone
>Priority: Critical
> Labels: azurecompute-arm
>
> Here the exception:
> {noformat}
> org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException:
> HTTP/1.1 429
> {}
> at
> org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:79)
> at
> org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:138)
> at
> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:107)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
> at
> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
> at
> org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> at
> com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:84)
> at com.sun.proxy.$Proxy267.getVersion
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.getImagesFromPublisher(AzureComputeServiceAdapter.java:212)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImagesByLocation(AzureComputeServiceAdapter.java:226)
> at
> org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImages(AzureComputeServiceAdapter.java:255)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:121)
> at
> org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:118)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:75)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:57)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
> at com.google.common.cache.LocalCache.get(LocalCache.java:4053)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
> at
> org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:150)
> at
> org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:106)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.getImages(TemplateBuilderImpl.java:853)
> at
> org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:665)
> {noformat}
> I use the Azure passing the imageId to template builder, I think that the
> cache is usefull in my case, but passing the override parameter:
> "jclouds.azurecompute.arm.publishers" to
