A great finding Jefrrey! Would this be OpenStack specific?
On Mon, May 19, 2014 at 12:08 PM, Jeffrey Nguyen (jeffrngu) < jeffr...@cisco.com> wrote: > Hi Nirmal, > > I did some further debugging on this issue and confirmed that the new > jClouds property mentioned below actually took effect. However, with or > without this new jClouds property enabled, the floating allocation process > fails when we invoke FloatinIPApi.create() method. This is due to a known > jClouds issue that has not been fixed according to > https://issues.apache.org/jira/browse/JCLOUDS-471. > > The problem is that at the Openstack level, when you create a floating > IP, you need to supply the name of the floating IP pool. I think create() > used to work with older version of Openstack where if floating IP pool info > is not supplied, Openstack will query the default configured value. > Apparently, this is no longer the case with Havana. > > To fix this issue, I used > FloatingIPApi.allocateFromPool(defaultFloatingIpPool) instead of create(). > This allows us to be more explicit as to which floating IP pool to use. > I'm introducing a new property named "defaultFloatingIpPool" in > cloud-controller.xml under IaasProvider section. This new property is > used as the default floating IP pool for the Iaas provider. If a > cartridge needs to override this, we can change the value in the cartridge > definition, under the IaasProvider section. > > I think not only the above will fix the current issue we're having with > floating IP allocation, it will also give us some flexibility in terms of > being able to explicitly specify which floating IP pool to be used. > > I made it so that If the new property is defined, I > use FloatingIPApi.allocateFromPool(). Otherwise, I use the existing > API FloatingIPApi.create(). I've tested and confirmed that with this fix, > the new instance is spawned and becomes active without the pre-allocated > floating IP. > > WDYT? > > -Jeffrey > > From: Nirmal Fernando <nir...@wso2.com> > Date: Monday, May 12, 2014 9:31 PM > To: jeffrngu <jeffr...@cisco.com> > Cc: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org> > > Subject: Re: Issue with floating IP allocation > > > > > On Tue, May 13, 2014 at 9:44 AM, Jeffrey Nguyen (jeffrngu) < > jeffr...@cisco.com> wrote: > >> Thank you Nirmal. Please see my responses under [Jeffrey] >> >> Also, in my previous email I had a question about why this new jClouds >> property along with the existing ones are not showing up in the payload for >> cartridge deployment. >> > > Jclouds wire log would not show these, these are used by CC and jclouds > for internal operations. These doesn't make sense to pass to the instance. > > >> I'm trying to confirm if the properties defined under Iaas provider >> are sent to jClouds. Can you take a look? >> >> Thanks, >> >> -Jeffrey >> >> From: Nirmal Fernando <nirmal070...@gmail.com> >> Reply-To: "dev@stratos.incubator.apache.org" < >> dev@stratos.incubator.apache.org> >> Date: Saturday, May 10, 2014 7:11 PM >> To: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org> >> Cc: Nirmal Fernando <nir...@wso2.com> >> Subject: Re: Issue with floating IP allocation >> >> Hi Jeffrey, >> >> >> On Fri, May 9, 2014 at 12:59 AM, Jeffrey Nguyen (jeffrngu) < >> jeffr...@cisco.com> wrote: >> >>> Copied the right email address… >>> >>> From: jeffrngu <jeffr...@cisco.com> >>> Date: Thursday, May 8, 2014 12:27 PM >>> To: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org>, >>> "Nirmal Fernando (JIRA)" <j...@apache.org> >>> Subject: Issue with floating IP allocation >>> >>> Hi Nirmal, >>> >>> Not sure when we resolve email issue with Stratos alias so I copied >>> you explicitly on this email. >>> Anyway, I opened STRATOS-646 yesterday to track this issue where >>> floating IP allocation fails when we don't pre-allocate the floating IP. >>> I was doing some experiment with a jClouds property named >>> "jclouds.openstack-nova.auto-create-floating-ips". I put it inside the >>> property section where we define the "autoAssignIp" parameter as shown in >>> [1] below. >>> When I tried to subscribe to this cartridge, I got some exceptions shown >>> in [2] below. >>> >> >> The place you have used this property is correct. But you have to set >> autoAssignIp too, to true. >> [Jeffrey] I tested again with autoAsignIp set to true in addition to >> the new jClouds property but I still saw the same problem. Any other >> suggestions? From the java-doc description for the jclouds property, it >> looks like it is used to create floating IP on-demand AND assign the same >> floating IP to the spawned instance. For our case, sometimes we want to >> assign our own floating IP, but we just need it to be allocated at the Iaas >> level without us manually creating it. I'm not sure this jClouds >> property was designed to handle this case. >> >> I remember Subbu had the same confusions and we managed to test it > successfully. > > >> >> >>> Please correct me if I'm wrong on this. Based on the implementation >>> for "ComputeServiceBuilderUtil.buildDefaultComputeService()", we're taking >>> the properties defined in the "property" section of cartridge definition >>> and send it to jClouds as-is. However, in the jClouds wire log, I don't >>> see them in the server create request being sent to jClouds. Or are they >>> encrypted inside "user_data"? >>> >>> Basically, I was trying to see if the above jClouds setting can solve >>> this issue with floating IP allocation. If it does, the next question >>> would be whether we use it as-is or we define a more user-friendly property >>> for Stratos and then internally mask it out with the actual jClouds >>> property. >>> >> >> Proper fix would be to set this property if and only if autoAssignIp >> set to true, in the OpenstackNovaIaas. Then we do not need to specify this >> jclouds property explicitly. Could you submit a patch? :-) >> [Jeffrey] I can work on this. However, I think we should make sure >> the new jClouds property can fix this issue before we worry about how to >> integrate it into Stratos code. >> >> >>> Please let me know what you think. >>> >>> Thanks, >>> -Jeffrey >>> >>> [1] Modified cartridge def >>> >>> "iaasProvider": [ >>> { >>> "imageId": "RegionOne/a00d6082-61c8-4a4c-be3d-9ffbf3968792", >>> "maxInstanceLimit": "4", >>> "networkInterfaces": [ >>> { >>> "networkUuid": >>> "de9464d6-a5c7-4e37-8842-602065a2546a", >>> } >>> ], >>> "property": [ >>> { >>> "name": "instanceType", >>> "value": >>> "RegionOne/a033a14f-ad30-4703-a5bd-ac6496e1d580" >>> }, >>> { >>> "name": "keyPair", >>> "value": "phoenix" >>> }, >>> { >>> "name": "autoAssignIp", >>> "value": "false" >>> }, >>> { >>> "name": " >>> *jclouds.openstack-nova.auto-create-floating-ips*", >>> "value": "true" >>> }, >>> { >>> "name": "securityGroups", >>> "value": "default" >>> } >>> ], >>> "type": "openstack-Core" >>> } >>> >>> >>> [2] Snippet from wso2carbon.log >>> >>> TID: [0] [STRATOS] [2014-05-08 10:44:02,184] DEBUG {jclouds.wire} - >>> >> >>> "{"server":{"name":"cisco-samp-980","imageRef":"a00d6082-61c8-4a4c-be3d-9ffbf3968792","flavorRef":"a033a14f-ad30-4703-a5bd-ac6496e1d580","metadata":{"jclouds-group":"cisco-samp"},"key_name":"phoenix","security_groups":[{"name":"default"}],"user_data":"U0VSVklDRV9OQU1FPWNpc2NvLXNhbXBsZS12bSxIT1NUX05BTUU9Y2lzY28tc2FtcGxlLXZtLnFtb2cuY2lzY28uY29tLE1VTFRJVEVOQU5UPWZhbHNlLFRFTkFOVF9JRD0tMTIzNCxURU5BTlRfUkFOR0U9LTEyMzQsQ0FSVFJJREdFX0FMSUFTPWNpc2NvLXNhbXBsZS12bSxDTFVTVEVSX0lEPWNpc2NvLXNhbXBsZS12bS5jaXNjby1zYW1wbGUtdixDQVJUUklER0VfS0VZPXBTQ1hCdmk2QXBxME5TY2MsREVQTE9ZTUVOVD1kZWZhdWx0LFJFUE9fVVJMPWh0dHA6Ly9vY3RsLTAxLnFtb2cuY2lzY28uY29tOjEwMDgwL2dpdC9kZWZhdWx0LmdpdCxQT1JUUz0yMixQVVBQRVRfSVA9UFVQUEVUX0lQLFBVUFBFVF9IT1NUTkFNRT1QVVBQRVRfSE9TVE5BTUUsUFVQUEVUX0VOVj1QVVBQRVRfRU5WLE1PTklUT1JJTkdfU0VSVkVSX0FETUlOX1VTRVJOQU1FPXh4eHgsTU9OSVRPUklOR19TRVJWRVJfU0VDVVJFX1BPUlQ9MCxDRVJUX1RSVVNUU1RPUkU9L29wdC9hcGFjaGUtc3RyYXRvcy1jYXJ0cmlkZ2UtYWdlbnQvc2VjdXJpdHkvY2xpZW50LXRydXN0c3RvcmUuamtzLE1PTklUT1JJTkdfU0VSVkVSX0lQPW9jdGwtMDEucW1vZy5jaXNjby5jb20sTUJfSVA9b2N0bC0wMS5xbW9nLmNpc2NvLmNvbSxNQl9QT1JUPTYxNjE2LENFUF9QT1JUPTc2MTEsTU9OSVRPUklOR19TRVJWRVJfQURNSU5fUEFTU1dPUkQ9eHh4eCxDRVBfSVA9b2N0bC0wMS5xbW9nLmNpc2NvLmNvbSxTSU1QTEVfUFJPUEVSVFk9dmFsdWUsTU9OSVRPUklOR19TRVJWRVJfUE9SVD0wLEVOQUJMRV9EQVRBX1BVQkxJU0hFUj1mYWxzZSxUUlVTVFNUT1JFX1BBU1NXT1JEPXdzbzJjYXJib24sQ09NTUlUX0VOQUJMRUQ9ZmFsc2UsTUVNQkVSX0lEPWNpc2NvLXNhbXBsZS12bS5jaXNjby1zYW1wbGUtdjgwZTY1ZDI5LTAzMmQtNGNkNy05NWQ1LTg1YmVjZGYyZTZjOCxMQl9DTFVTVEVSX0lEPW51bGwsTkVUV09SS19QQVJUSVRJT05fSUQ9TjEsUEFSVElUSU9OX0lEPVJlZ2lvbk9uZQ\u003d\u003d","networks":[{"uuid":"de9464d6-a5c7-4e37-8842-602065a2546a"}]}}" >>> {jclouds.wire} >>> TID: [0] [STRATOS] [2014-05-08 10:44:02,707] DEBUG {jclouds.wire} - << >>> "{"server": {"security_groups": [{"name": "default"}], "OS-DCF:diskConfig": >>> "MANUAL", "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", "links": [{"href": " >>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "self"}, {"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "bookmark"}], "adminPass": "P6BA3jPCPPzB"}}" {jclouds.wire} >>> TID: [0] [STRATOS] [2014-05-08 10:44:02,854] DEBUG {jclouds.wire} - << >>> "{"server": {"status": "BUILD", "updated": "2014-05-08T10:44:02Z", >>> "hostId": "", "OS-EXT-SRV-ATTR:host": null, "addresses": {}, "links": >>> [{"href": " >>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "self"}, {"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "bookmark"}], "key_name": "phoenix", "image": {"id": >>> "a00d6082-61c8-4a4c-be3d-9ffbf3968792", "links": [{"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/images/a00d6082-61c8-4a4c-be3d-9ffbf3968792";, >>> "rel": "bookmark"}]}, "OS-EXT-STS:task_state": "scheduling", >>> "OS-EXT-STS:vm_state": "building", "OS-EXT-SRV-ATTR:instance_name": >>> "instance-000001af", "OS-SRV-USG:launched_at": null, >>> "OS-EXT-SRV-ATTR:hypervisor_hostname": null, "flavor": {"id": >>> "a033a14f-ad30-4703-a5bd-ac6496e1d580", "links": [{"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/flavors/a033a14f-ad30-4703-a5bd-ac6496e1d580";, >>> "rel": "bookmark"}]}, "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", >>> "OS-SRV-USG:terminated_at": null, "OS-EXT-AZ:availability_zone": "nova", >>> "user_id": "bfb57e32b9c941608112db017ba9248f", "name": "cisco-samp-980", >>> "created": "2014-05-08T10:44:02Z", "tenant_id": >>> "161d4e9387704433a8519259d65186ba", "OS-DCF:diskConfig": "MANUAL", >>> "os-extended-volumes:volumes_attached": [], "accessIPv4": "", "accessIPv6": >>> "", "progress": 0, "OS-EXT-STS:power_state": 0, "config_drive": "", >>> "metadata": {"jclouds-group": "cisco-samp"}}}" {jclouds.wire} >>> …. >>> TID: [0] [STRATOS] [2014-05-08 10:44:07,953] DEBUG {jclouds.wire} - >>> << "{"server": {"status": "ACTIVE", "updated": "2014-05-08T10:44:07Z", >>> "hostId": "30b72df643a1170bf586d61c107d73c5efee42f812e95f6c4a98663d", >>> "OS-EXT-SRV-ATTR:host": "denver-stack03", "addresses": {"core": >>> [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:ac:28:59", "version": 4, "addr": >>> "172.16.2.17", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": " >>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "self"}, {"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>> "rel": "bookmark"}], "key_name": "phoenix", "image": {"id": >>> "a00d6082-61c8-4a4c-be3d-9ffbf3968792", "links": [{"href": " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/images/a00d6082-61c8-4a4c-be3d-9ffbf3968792";, >>> "rel": "bookmark"}]}, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": >>> "active", "OS-EXT-SRV-ATTR:instance_name": "instance-000001af", >>> "OS-SRV-USG:launched_at": "2014-05-08T10:44:06.000000", >>> "OS-EXT-SRV-ATTR:hypervisor_hostname": "denver-stack03.cisco.com", >>> "flavor": {"id": "a033a14f-ad30-4703-a5bd-ac6496e1d580", "links": [{"href": >>> " >>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/flavors/a033a14f-ad30-4703-a5bd-ac6496e1d580";, >>> "rel": "bookmark"}]}, "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", >>> "security_groups": [{"name": "default"}], "OS-SRV-USG:terminated_at": null, >>> "OS-EXT-AZ:availability_zone": "AZ-1", "user_id": >>> "bfb57e32b9c941608112db017ba9248f", "name": "cisco-samp-980", "created": >>> "2014-05-08T10:44:02Z", "tenant_id": "161d4e9387704433a8519259d65186ba", >>> "OS-DCF:diskConfig": "MANUAL", "os-extended-volumes:volumes_attached": [], >>> "accessIPv4": "", "accessIPv6": "", "progress": 0, >>> "OS-EXT-STS:power_state": 1, "config_drive": "", "metadata": >>> {"jclouds-group": "cisco-samp"}}}" {jclouds.wire} >>> TID: [0] [STRATOS] [2014-05-08 10:44:07,965] DEBUG {jclouds.wire} - >> >>> "{}" {jclouds.wire} >>> TID: [0] [STRATOS] [2014-05-08 10:44:07,982] DEBUG {jclouds.wire} - << >>> "{"itemNotFound": {"message": "FloatingIpPoolNotFound: Floating ip pool not >>> found.", "code": 404}}" {jclouds.wire} >>> TID: [0] [STRATOS] [2014-05-08 10:44:07,987] ERROR {jclouds.compute} - >>> createNodesInGroup(cisco-samp), completed: 0/1, errors: 1, rate: 5816ms/op >>> {jclouds.compute} >>> java.util.concurrent.ExecutionException: java.lang.NullPointerException >>> at >>> com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) >>> at >>> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) >>> at >>> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) >>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:125) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.lang.NullPointerException >>> at >>> org.jclouds.openstack.nova.v2_0.compute.functions.AllocateAndAddFloatingIpToNode.apply(AllocateAndAddFloatingIpToNode.java:100) >>> at >>> org.jclouds.openstack.nova.v2_0.compute.functions.AllocateAndAddFloatingIpToNode.apply(AllocateAndAddFloatingIpToNode.java:53) >>> at com.google.common.util.concurrent.Futures$1.apply(Futures.java:711) >>> at >>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:849) >>> ... 3 more >>> TID: [0] [STRATOS] [2014-05-08 10:44:07,988] ERROR {jclouds.compute} - >>> createNodesInGroup(cisco-samp), completed: 0/1, errors: 1, rate: 5817ms/op >>> {jclouds.compute} >>> java.lang.RuntimeException: createNodesInGroup(cisco-samp), completed: >>> 0/1, errors: 1, rate: 5817ms/op >>> at >>> org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:155) >>> at >>> org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:219) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37) >>> at com.sun.proxy.$Proxy91.createNodesInGroup(Unknown Source) >>> at >>> org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl.startInstance(CloudControllerServiceImpl.java:356) >>> >>> >>> >> >> >> -- >> Best Regards, >> Nirmal >> >> Nirmal Fernando. >> PPMC Member & Committer of Apache Stratos, >> Senior Software Engineer, WSO2 Inc. >> >> Blog: http://nirmalfdo.blogspot.com/ >> > > > > -- > > Thanks & regards, > Nirmal > > Senior Software Engineer- Platform Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- Imesh Gunaratne Technical Lead, WSO2 Committer & PPMC Member, Apache Stratos
