Simon Hildrew created JCLOUDS-490:
-------------------------------------
Summary: Security group listing on nova slow
Key: JCLOUDS-490
URL: https://issues.apache.org/jira/browse/JCLOUDS-490
Project: jclouds
Issue Type: Bug
Affects Versions: 1.7.1, 1.7.0
Environment: Linux, Scala, Openstack
Reporter: Simon Hildrew
Calls to retrieve the security groups in an Openstack tenant using the jClouds
ComputeService API is inefficient and very slow.
The following (scala) code is an example of a slow call:
{noformat}
val context = ContextBuilder.newBuilder("openstack-nova")
.endpoint(origin.endpoint)
.credentials(s"${origin.tenant}:${origin.user}", origin.secret)
.build(classOf[ComputeServiceContext])
val compute = context.getComputeService
val securityGroupExtension = compute.getSecurityGroupExtension.get
val secGroups = securityGroupExtension.listSecurityGroups
val result = secGroups.map ( fromJCloud(_, existingGroups) )
{noformat}
Most of this is very quick, but when debugging I the call to listSecurityGroups
is slow. In one example run this took 6 minutes and 17 seconds to retrieve 32
security groups. During this example call, 93 separate requests are made to the
Openstack API server. The first is a call to GET /v2/<id>/extensions and the
following 92 are calls to GET /v2/<id>/os-security-groups. These 92 calls all
get an identical response from the API and the calls are made with about 4
seconds delay. After all these calls the method returns with the correct
response.
One of the causes (but this doesn't seem to account for all the the excess
calls I'm seeing) is that the NovaSecurityGroupToSecurityGroup class makes
calls to transform each openstack SecurityGroupRule into a JClouds IpPermission
object. This is done in the SecurityGroupRuleToIpPermission class, which takes
a Predicate<AtomicReference<ZoneAndName>> and the supplied predicate is called
for every transformation that occurs in which the source is another security
group instead of a CIDR. The implementation of the predicate is
FindSecurityGroupWithNameAndReturnTrue - which makes an API request for the
list of security groups in a zone and then returns true if the supplied
reference is in the newly obtained list. The API call is made regardless of
whether it is the same zone or not.
In the particular zone I'm querying, there are a large number of rules (60)
that reference other security groups (all in the same zone). As I said, this
doesn't account for all the calls for exactly the same request - but it does
account for more than half of them. I think there must be another
transformation that also uses the predicate.
I note that the predicate actually modifies the object, which makes it trickier
to remove without changing the behaviour.
As a work around I am now using the NovaApi directly and can report that it
takes a few seconds, instead of the 6 minutes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
