I don't think adding an API to CloudStack is particularly appropriate way to integrate with whirr? Whirr should *use* the CloudStack API, not add to it?
On 9/20/13 1:09 AM, "Sebastien Goasguen" <run...@gmail.com> wrote: > >On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote: > >> >> The Nicira NVP plugin is also depending on gson. If we make any changes >>we need to validate against their api to ensure that the interface still >>works. >> >> If we put the changes in a separate branch i'd be happy to run the >>changes against the api. >> >> Cheers, >> >> Hugo > >So how can Meng get out of this bind, she is trying to complete her >google summer of code project. > >thanks, > >-Sebastien > >> >> >> On Sep 20, 2013, at 12:08 PM, Darren Shepherd >><darren.s.sheph...@gmail.com> wrote: >> >>> After much searching I found the two lines of code I needed to get it >>>to serialize right. I'll keep looking and see if I find other snags. >>>Maybe we can just move to jackson.... we'll see, haven't tried to >>>deserialize yet. >>> >>> Darren >>> >>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> wrote: >>>> >>>> Darren, >>>> >>>> When I looked into updating to the latest gson, the problem, IIRC, is >>>>things that weren't considered cyclical dependencies are suddenly >>>>considered cyclical with the latest. I don't recall where though. >>>> >>>> --Alex >>>> >>>>> -----Original Message----- >>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>>> Sent: Thursday, September 19, 2013 1:33 PM >>>>> To: dev@cloudstack.apache.org >>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>> >>>>> Alright, I looked into this and it will take a bit more work to >>>>>switch to Jackson. >>>>> The snag I hit was how we serialize commands for the agents. We use >>>>>a >>>>> customer typeadatper in gson to produce a format like { "org.MyClass" >>>>> : {...} }. In jackson I don't see producing a similar format as >>>>>being so straight >>>>> forward. I'm sure there's an elegant solution, but I didn't >>>>>immediately find it. >>>>> The problem is if you use a custom serializer in jackson and do >>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a >>>>>loop >>>>> because it will call your serializer again. So if somebody can >>>>>figure out how to >>>>> do this format easily in jackson, the rest looks simple enough. >>>>> >>>>> So, being that it is not an obvious switch to using jackson, what is >>>>>the impact >>>>> of moving to the latest gson? What were the issue encountered when >>>>> people tried it before? >>>>> >>>>> Darren >>>>> >>>>> >>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>>> darren.s.sheph...@gmail.com> wrote: >>>>> >>>>>> I can do some analysis on this. I'm always up for a terribly >>>>>>painful >>>>>> refactor :) >>>>>> >>>>>> Darren >>>>>> >>>>>> >>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <alex.hu...@citrix.com> >>>>>> wrote: >>>>>> >>>>>>> I actually did a quick try to update cloudstack to use newest gson >>>>>>> version about 3 months back. Had to roll it back but I didn't try >>>>>>> very hard though. Part of the reason why I decided to rollback is >>>>>>> due to gson is used differently by various components in CloudStack >>>>>>> and I didn't have time to get into each component to see if I break >>>>>>> anything. The other is also I thought using Jackson would be much >>>>>>> better and thought our next attempt should be with Jackson. >>>>>>> >>>>>>> Not that any of these helps you. Just know updating CloudStack to >>>>>>> latest gson is not simple. >>>>>>> >>>>>>> --Alex >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>>> To: d...@whirr.apache.org >>>>>>>> Cc: dev@cloudstack.apache.org >>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>>> >>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just >>>>>>>> add >>>>>>> the >>>>>>>> feature url. Bonus: you can also use jclouds shell interface (part >>>>>>>> of >>>>>>> jclouds cli). >>>>>>>> >>>>>>>> Another option is to upgrade the CloudStack API to use the new >>>>>>>>version. >>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>>> >>>>>>>>> Dear all, >>>>>>>>> >>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>>> conflicting >>>>>>>> issue. >>>>>>>>> >>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>>> gson >>>>>>> 1.7.1. >>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>>> >>>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>>> com/google/gson/TypeAdapter >>>>>>>>> >>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will >>>>>>>>> not build successfully due to the following test error. >>>>>>>>> >>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>>> ------------------------------**------------------------------** >>>>>>>>> ------------ >>>>>>>>> [INFO] >>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>>> cloud-core >>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>>> [INFO] >>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) >>>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8' >>>>>>>>> encoding to copy filtered resources. >>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>>> 3 >>>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile >>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered >>>>>>>>>resources. >>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>>> 3 >>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source >>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO] >>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>>> cloud-core >>>>>>>>> --- >>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>>> target/surefire-reports >>>>>>>>> >>>>>>>>> ------------------------------**------------------------- >>>>>>>>> T E S T S >>>>>>>>> ------------------------------**------------------------- >>>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>>> log4j:WARN See >>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>>> http://logging.apa >>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>>> >>>>>>>>> Results : >>>>>>>>> >>>>>>>>> Failed tests: >>>>>>>>>testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>>> >>>>>>>>> Tests in error: >>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>>> >>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>>> >>>>>>>>> >>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>>> API --launchCluster to CloudStack, the following error related to >>>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>>> >>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught >>>>>>>>> problem with >>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>>> >>>>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy" >>>>>>>>> :"** >>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>> **LibvirtComputingResource","* >>>>>>>>> * >>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>>> 10.244.18 >>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-* >>>>>>>>> * >>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource >>>>>>>>> ","** >>>>>>>>> contextMap":{},"wait":0}}] >>>>>>>>> java.lang.ClassCastException: >>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>>> cannot be cast to java.lang.Class >>>>>>>>> at >>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>>> at >>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>>> TreeTypeAdapter.java:58) >>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>>> at >>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>>> java:235) >>>>>>>>> at >>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>>> at >>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>>> ) >>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>>> at >>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>>> following exception but pushing on >>>>>>>>> >>>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>>> then quickly died. >>>>>>>>> Running on provider cloudstack using identity >>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>>> ZRGXVO1VApG >>>>>>>>> 6q0a >>>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:) >>>>>>>>> Configuring template for >>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>>> (729061754@qtp-385354117- >>>>>>>> 0:) >>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>>> jobtracker >>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) >>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>>> >>>>>>>>> I attached the debuging log to this email. >>>>>>>>> >>>>>>>>> The launchCluster API is simple, it calls the >>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>>> >>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>>> LaunchClusterCommand command = null; >>>>>>>>> try { >>>>>>>>> command = new LaunchClusterCommand(); >>>>>>>>> } catch (Exception ex) { >>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>> VER >>>>>>>>> E, >>>>>>>>> null, ex); >>>>>>>>> } >>>>>>>>> String[] args = new String[2]; >>>>>>>>> args[0] = "--config"; >>>>>>>>> args[1] = config; >>>>>>>>> >>>>>>>>> try { >>>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>>> Arrays.asList(args)); >>>>>>>>> } catch (Exception ex) { >>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>> VER >>>>>>>>> E, >>>>>>>>> null, ex); >>>>>>>>> } >>>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>>> output = "successfully launched the cluster."; >>>>>>>>> response.setOutPut(output); >>>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>>> this.setResponseObject(**response); >>>>>>>>> >>>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>>> >>>>>>>>> Thanks loads. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> Meng >>>>>> >>>>>> >> >
