Great! May be we could document this. Thanks
On Wed, May 21, 2014 at 10:10 AM, Jeffrey Nguyen (jeffrngu) < jeffr...@cisco.com> wrote: > Hi Imesh, > > Yes, this is OpenStack specific. > > -Jeffrey > > From: Imesh Gunaratne <im...@apache.org> > Reply-To: "dev@stratos.incubator.apache.org" < > dev@stratos.incubator.apache.org> > Date: Tuesday, May 20, 2014 9:32 PM > > To: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org> > Cc: Nirmal Fernando <nir...@wso2.com> > Subject: Re: Issue with floating IP allocation > > A great finding Jefrrey! Would this be OpenStack specific? > > > On Mon, May 19, 2014 at 12:08 PM, Jeffrey Nguyen (jeffrngu) < > jeffr...@cisco.com> wrote: > >> Hi Nirmal, >> >> I did some further debugging on this issue and confirmed that the new >> jClouds property mentioned below actually took effect. However, with or >> without this new jClouds property enabled, the floating allocation process >> fails when we invoke FloatinIPApi.create() method. This is due to a known >> jClouds issue that has not been fixed according to >> https://issues.apache.org/jira/browse/JCLOUDS-471. >> >> The problem is that at the Openstack level, when you create a floating >> IP, you need to supply the name of the floating IP pool. I think create() >> used to work with older version of Openstack where if floating IP pool info >> is not supplied, Openstack will query the default configured value. >> Apparently, this is no longer the case with Havana. >> >> To fix this issue, I used >> FloatingIPApi.allocateFromPool(defaultFloatingIpPool) instead of create(). >> This allows us to be more explicit as to which floating IP pool to use. >> I'm introducing a new property named "defaultFloatingIpPool" in >> cloud-controller.xml under IaasProvider section. This new property is >> used as the default floating IP pool for the Iaas provider. If a >> cartridge needs to override this, we can change the value in the cartridge >> definition, under the IaasProvider section. >> >> I think not only the above will fix the current issue we're having with >> floating IP allocation, it will also give us some flexibility in terms of >> being able to explicitly specify which floating IP pool to be used. >> >> I made it so that If the new property is defined, I >> use FloatingIPApi.allocateFromPool(). Otherwise, I use the existing >> API FloatingIPApi.create(). I've tested and confirmed that with this fix, >> the new instance is spawned and becomes active without the pre-allocated >> floating IP. >> >> WDYT? >> >> -Jeffrey >> >> From: Nirmal Fernando <nir...@wso2.com> >> Date: Monday, May 12, 2014 9:31 PM >> To: jeffrngu <jeffr...@cisco.com> >> Cc: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org> >> >> >> Subject: Re: Issue with floating IP allocation >> >> >> >> >> On Tue, May 13, 2014 at 9:44 AM, Jeffrey Nguyen (jeffrngu) < >> jeffr...@cisco.com> wrote: >> >>> Thank you Nirmal. Please see my responses under [Jeffrey] >>> >>> Also, in my previous email I had a question about why this new jClouds >>> property along with the existing ones are not showing up in the payload for >>> cartridge deployment. >>> >> >> Jclouds wire log would not show these, these are used by CC and jclouds >> for internal operations. These doesn't make sense to pass to the instance. >> >> >>> I'm trying to confirm if the properties defined under Iaas provider >>> are sent to jClouds. Can you take a look? >>> >>> Thanks, >>> >>> -Jeffrey >>> >>> From: Nirmal Fernando <nirmal070...@gmail.com> >>> Reply-To: "dev@stratos.incubator.apache.org" < >>> dev@stratos.incubator.apache.org> >>> Date: Saturday, May 10, 2014 7:11 PM >>> To: "dev@stratos.incubator.apache.org" <dev@stratos.incubator.apache.org >>> > >>> Cc: Nirmal Fernando <nir...@wso2.com> >>> Subject: Re: Issue with floating IP allocation >>> >>> Hi Jeffrey, >>> >>> >>> On Fri, May 9, 2014 at 12:59 AM, Jeffrey Nguyen (jeffrngu) < >>> jeffr...@cisco.com> wrote: >>> >>>> Copied the right email address… >>>> >>>> From: jeffrngu <jeffr...@cisco.com> >>>> Date: Thursday, May 8, 2014 12:27 PM >>>> To: "dev@stratos.incubator.apache.org" < >>>> dev@stratos.incubator.apache.org>, "Nirmal Fernando (JIRA)" < >>>> j...@apache.org> >>>> Subject: Issue with floating IP allocation >>>> >>>> Hi Nirmal, >>>> >>>> Not sure when we resolve email issue with Stratos alias so I copied >>>> you explicitly on this email. >>>> Anyway, I opened STRATOS-646 yesterday to track this issue where >>>> floating IP allocation fails when we don't pre-allocate the floating IP. >>>> I was doing some experiment with a jClouds property named >>>> "jclouds.openstack-nova.auto-create-floating-ips". I put it inside the >>>> property section where we define the "autoAssignIp" parameter as shown in >>>> [1] below. >>>> When I tried to subscribe to this cartridge, I got some exceptions >>>> shown in [2] below. >>>> >>> >>> The place you have used this property is correct. But you have to set >>> autoAssignIp too, to true. >>> [Jeffrey] I tested again with autoAsignIp set to true in addition >>> to the new jClouds property but I still saw the same problem. Any other >>> suggestions? From the java-doc description for the jclouds property, it >>> looks like it is used to create floating IP on-demand AND assign the same >>> floating IP to the spawned instance. For our case, sometimes we want to >>> assign our own floating IP, but we just need it to be allocated at the Iaas >>> level without us manually creating it. I'm not sure this jClouds >>> property was designed to handle this case. >>> >>> I remember Subbu had the same confusions and we managed to test it >> successfully. >> >> >>> >>> >>>> Please correct me if I'm wrong on this. Based on the implementation >>>> for "ComputeServiceBuilderUtil.buildDefaultComputeService()", we're taking >>>> the properties defined in the "property" section of cartridge definition >>>> and send it to jClouds as-is. However, in the jClouds wire log, I don't >>>> see them in the server create request being sent to jClouds. Or are they >>>> encrypted inside "user_data"? >>>> >>>> Basically, I was trying to see if the above jClouds setting can solve >>>> this issue with floating IP allocation. If it does, the next question >>>> would be whether we use it as-is or we define a more user-friendly property >>>> for Stratos and then internally mask it out with the actual jClouds >>>> property. >>>> >>> >>> Proper fix would be to set this property if and only if autoAssignIp >>> set to true, in the OpenstackNovaIaas. Then we do not need to specify this >>> jclouds property explicitly. Could you submit a patch? :-) >>> [Jeffrey] I can work on this. However, I think we should make sure >>> the new jClouds property can fix this issue before we worry about how to >>> integrate it into Stratos code. >>> >>> >>>> Please let me know what you think. >>>> >>>> Thanks, >>>> -Jeffrey >>>> >>>> [1] Modified cartridge def >>>> >>>> "iaasProvider": [ >>>> { >>>> "imageId": >>>> "RegionOne/a00d6082-61c8-4a4c-be3d-9ffbf3968792", >>>> "maxInstanceLimit": "4", >>>> "networkInterfaces": [ >>>> { >>>> "networkUuid": >>>> "de9464d6-a5c7-4e37-8842-602065a2546a", >>>> } >>>> ], >>>> "property": [ >>>> { >>>> "name": "instanceType", >>>> "value": >>>> "RegionOne/a033a14f-ad30-4703-a5bd-ac6496e1d580" >>>> }, >>>> { >>>> "name": "keyPair", >>>> "value": "phoenix" >>>> }, >>>> { >>>> "name": "autoAssignIp", >>>> "value": "false" >>>> }, >>>> { >>>> "name": " >>>> *jclouds.openstack-nova.auto-create-floating-ips*", >>>> "value": "true" >>>> }, >>>> { >>>> "name": "securityGroups", >>>> "value": "default" >>>> } >>>> ], >>>> "type": "openstack-Core" >>>> } >>>> >>>> >>>> [2] Snippet from wso2carbon.log >>>> >>>> TID: [0] [STRATOS] [2014-05-08 10:44:02,184] DEBUG {jclouds.wire} - >>>> >> >>>> "{"server":{"name":"cisco-samp-980","imageRef":"a00d6082-61c8-4a4c-be3d-9ffbf3968792","flavorRef":"a033a14f-ad30-4703-a5bd-ac6496e1d580","metadata":{"jclouds-group":"cisco-samp"},"key_name":"phoenix","security_groups":[{"name":"default"}],"user_data":"U0VSVklDRV9OQU1FPWNpc2NvLXNhbXBsZS12bSxIT1NUX05BTUU9Y2lzY28tc2FtcGxlLXZtLnFtb2cuY2lzY28uY29tLE1VTFRJVEVOQU5UPWZhbHNlLFRFTkFOVF9JRD0tMTIzNCxURU5BTlRfUkFOR0U9LTEyMzQsQ0FSVFJJREdFX0FMSUFTPWNpc2NvLXNhbXBsZS12bSxDTFVTVEVSX0lEPWNpc2NvLXNhbXBsZS12bS5jaXNjby1zYW1wbGUtdixDQVJUUklER0VfS0VZPXBTQ1hCdmk2QXBxME5TY2MsREVQTE9ZTUVOVD1kZWZhdWx0LFJFUE9fVVJMPWh0dHA6Ly9vY3RsLTAxLnFtb2cuY2lzY28uY29tOjEwMDgwL2dpdC9kZWZhdWx0LmdpdCxQT1JUUz0yMixQVVBQRVRfSVA9UFVQUEVUX0lQLFBVUFBFVF9IT1NUTkFNRT1QVVBQRVRfSE9TVE5BTUUsUFVQUEVUX0VOVj1QVVBQRVRfRU5WLE1PTklUT1JJTkdfU0VSVkVSX0FETUlOX1VTRVJOQU1FPXh4eHgsTU9OSVRPUklOR19TRVJWRVJfU0VDVVJFX1BPUlQ9MCxDRVJUX1RSVVNUU1RPUkU9L29wdC9hcGFjaGUtc3RyYXRvcy1jYXJ0cmlkZ2UtYWdlbnQvc2VjdXJpdHkvY2xpZW50LXRydXN0c3RvcmUuamtzLE1PTklUT1JJTkdfU0VSVkVSX0lQPW9jdGwtMDEucW1vZy5jaXNjby5jb20sTUJfSVA9b2N0bC0wMS5xbW9nLmNpc2NvLmNvbSxNQl9QT1JUPTYxNjE2LENFUF9QT1JUPTc2MTEsTU9OSVRPUklOR19TRVJWRVJfQURNSU5fUEFTU1dPUkQ9eHh4eCxDRVBfSVA9b2N0bC0wMS5xbW9nLmNpc2NvLmNvbSxTSU1QTEVfUFJPUEVSVFk9dmFsdWUsTU9OSVRPUklOR19TRVJWRVJfUE9SVD0wLEVOQUJMRV9EQVRBX1BVQkxJU0hFUj1mYWxzZSxUUlVTVFNUT1JFX1BBU1NXT1JEPXdzbzJjYXJib24sQ09NTUlUX0VOQUJMRUQ9ZmFsc2UsTUVNQkVSX0lEPWNpc2NvLXNhbXBsZS12bS5jaXNjby1zYW1wbGUtdjgwZTY1ZDI5LTAzMmQtNGNkNy05NWQ1LTg1YmVjZGYyZTZjOCxMQl9DTFVTVEVSX0lEPW51bGwsTkVUV09SS19QQVJUSVRJT05fSUQ9TjEsUEFSVElUSU9OX0lEPVJlZ2lvbk9uZQ\u003d\u003d","networks":[{"uuid":"de9464d6-a5c7-4e37-8842-602065a2546a"}]}}" >>>> {jclouds.wire} >>>> TID: [0] [STRATOS] [2014-05-08 10:44:02,707] DEBUG {jclouds.wire} - << >>>> "{"server": {"security_groups": [{"name": "default"}], "OS-DCF:diskConfig": >>>> "MANUAL", "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", "links": [{"href": >>>> " >>>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "self"}, {"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "bookmark"}], "adminPass": "P6BA3jPCPPzB"}}" {jclouds.wire} >>>> TID: [0] [STRATOS] [2014-05-08 10:44:02,854] DEBUG {jclouds.wire} - << >>>> "{"server": {"status": "BUILD", "updated": "2014-05-08T10:44:02Z", >>>> "hostId": "", "OS-EXT-SRV-ATTR:host": null, "addresses": {}, "links": >>>> [{"href": " >>>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "self"}, {"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "bookmark"}], "key_name": "phoenix", "image": {"id": >>>> "a00d6082-61c8-4a4c-be3d-9ffbf3968792", "links": [{"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/images/a00d6082-61c8-4a4c-be3d-9ffbf3968792";, >>>> "rel": "bookmark"}]}, "OS-EXT-STS:task_state": "scheduling", >>>> "OS-EXT-STS:vm_state": "building", "OS-EXT-SRV-ATTR:instance_name": >>>> "instance-000001af", "OS-SRV-USG:launched_at": null, >>>> "OS-EXT-SRV-ATTR:hypervisor_hostname": null, "flavor": {"id": >>>> "a033a14f-ad30-4703-a5bd-ac6496e1d580", "links": [{"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/flavors/a033a14f-ad30-4703-a5bd-ac6496e1d580";, >>>> "rel": "bookmark"}]}, "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", >>>> "OS-SRV-USG:terminated_at": null, "OS-EXT-AZ:availability_zone": "nova", >>>> "user_id": "bfb57e32b9c941608112db017ba9248f", "name": "cisco-samp-980", >>>> "created": "2014-05-08T10:44:02Z", "tenant_id": >>>> "161d4e9387704433a8519259d65186ba", "OS-DCF:diskConfig": "MANUAL", >>>> "os-extended-volumes:volumes_attached": [], "accessIPv4": "", "accessIPv6": >>>> "", "progress": 0, "OS-EXT-STS:power_state": 0, "config_drive": "", >>>> "metadata": {"jclouds-group": "cisco-samp"}}}" {jclouds.wire} >>>> …. >>>> TID: [0] [STRATOS] [2014-05-08 10:44:07,953] DEBUG {jclouds.wire} - >>>> << "{"server": {"status": "ACTIVE", "updated": "2014-05-08T10:44:07Z", >>>> "hostId": "30b72df643a1170bf586d61c107d73c5efee42f812e95f6c4a98663d", >>>> "OS-EXT-SRV-ATTR:host": "denver-stack03", "addresses": {"core": >>>> [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:ac:28:59", "version": 4, "addr": >>>> "172.16.2.17", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": " >>>> http://10.94.251.12:8774/v2/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "self"}, {"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/servers/f0a4a1ae-1a7f-418f-a629-423a1f5ff998";, >>>> "rel": "bookmark"}], "key_name": "phoenix", "image": {"id": >>>> "a00d6082-61c8-4a4c-be3d-9ffbf3968792", "links": [{"href": " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/images/a00d6082-61c8-4a4c-be3d-9ffbf3968792";, >>>> "rel": "bookmark"}]}, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": >>>> "active", "OS-EXT-SRV-ATTR:instance_name": "instance-000001af", >>>> "OS-SRV-USG:launched_at": "2014-05-08T10:44:06.000000", >>>> "OS-EXT-SRV-ATTR:hypervisor_hostname": "denver-stack03.cisco.com", >>>> "flavor": {"id": "a033a14f-ad30-4703-a5bd-ac6496e1d580", "links": [{"href": >>>> " >>>> http://10.94.251.12:8774/161d4e9387704433a8519259d65186ba/flavors/a033a14f-ad30-4703-a5bd-ac6496e1d580";, >>>> "rel": "bookmark"}]}, "id": "f0a4a1ae-1a7f-418f-a629-423a1f5ff998", >>>> "security_groups": [{"name": "default"}], "OS-SRV-USG:terminated_at": null, >>>> "OS-EXT-AZ:availability_zone": "AZ-1", "user_id": >>>> "bfb57e32b9c941608112db017ba9248f", "name": "cisco-samp-980", "created": >>>> "2014-05-08T10:44:02Z", "tenant_id": "161d4e9387704433a8519259d65186ba", >>>> "OS-DCF:diskConfig": "MANUAL", "os-extended-volumes:volumes_attached": [], >>>> "accessIPv4": "", "accessIPv6": "", "progress": 0, >>>> "OS-EXT-STS:power_state": 1, "config_drive": "", "metadata": >>>> {"jclouds-group": "cisco-samp"}}}" {jclouds.wire} >>>> TID: [0] [STRATOS] [2014-05-08 10:44:07,965] DEBUG {jclouds.wire} - >> >>>> "{}" {jclouds.wire} >>>> TID: [0] [STRATOS] [2014-05-08 10:44:07,982] DEBUG {jclouds.wire} - << >>>> "{"itemNotFound": {"message": "FloatingIpPoolNotFound: Floating ip pool not >>>> found.", "code": 404}}" {jclouds.wire} >>>> TID: [0] [STRATOS] [2014-05-08 10:44:07,987] ERROR {jclouds.compute} - >>>> createNodesInGroup(cisco-samp), completed: 0/1, errors: 1, rate: 5816ms/op >>>> {jclouds.compute} >>>> java.util.concurrent.ExecutionException: java.lang.NullPointerException >>>> at >>>> com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) >>>> at >>>> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) >>>> at >>>> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) >>>> at >>>> org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:125) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>> at java.lang.Thread.run(Thread.java:744) >>>> Caused by: java.lang.NullPointerException >>>> at >>>> org.jclouds.openstack.nova.v2_0.compute.functions.AllocateAndAddFloatingIpToNode.apply(AllocateAndAddFloatingIpToNode.java:100) >>>> at >>>> org.jclouds.openstack.nova.v2_0.compute.functions.AllocateAndAddFloatingIpToNode.apply(AllocateAndAddFloatingIpToNode.java:53) >>>> at com.google.common.util.concurrent.Futures$1.apply(Futures.java:711) >>>> at >>>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:849) >>>> ... 3 more >>>> TID: [0] [STRATOS] [2014-05-08 10:44:07,988] ERROR {jclouds.compute} - >>>> createNodesInGroup(cisco-samp), completed: 0/1, errors: 1, rate: 5817ms/op >>>> {jclouds.compute} >>>> java.lang.RuntimeException: createNodesInGroup(cisco-samp), completed: >>>> 0/1, errors: 1, rate: 5817ms/op >>>> at >>>> org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:155) >>>> at >>>> org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:219) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>> at >>>> com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37) >>>> at com.sun.proxy.$Proxy91.createNodesInGroup(Unknown Source) >>>> at >>>> org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl.startInstance(CloudControllerServiceImpl.java:356) >>>> >>>> >>>> >>> >>> >>> -- >>> Best Regards, >>> Nirmal >>> >>> Nirmal Fernando. >>> PPMC Member & Committer of Apache Stratos, >>> Senior Software Engineer, WSO2 Inc. >>> >>> Blog: http://nirmalfdo.blogspot.com/ >>> >> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > Imesh Gunaratne > > Technical Lead, WSO2 > Committer & PPMC Member, Apache Stratos > -- Imesh Gunaratne Technical Lead, WSO2 Committer & PPMC Member, Apache Stratos
