On Sep 23, 2013, at 7:12 AM, Darren Shepherd <darren.s.sheph...@gmail.com> wrote:
> The only way this will get committed is if we either move to latest gson or > jackson. Regardless, the outcome will be that Meng can assume this gson > conflict won't happen. The problem is just how fast can we move all of ACS > to a new json library. Is it possible for Meng to commit to a branch that > just blindly updates gson to the latest (in /pom.xml). Yes I will talk to Meng about this. thanks > When we get ACS on > a newer json library, we can merge in that branch. Tomorrow (9/23), I can > attempt to put together a patch to move to jackson. I think I ironed out > most of the technical issues so I just have to globally apply the changes. > > Darren > > > On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen <run...@gmail.com>wrote: > >> >> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote: >> >>> >>> The Nicira NVP plugin is also depending on gson. If we make any changes >> we need to validate against their api to ensure that the interface still >> works. >>> >>> If we put the changes in a separate branch i'd be happy to run the >> changes against the api. >>> >>> Cheers, >>> >>> Hugo >> >> So how can Meng get out of this bind, she is trying to complete her google >> summer of code project. >> >> thanks, >> >> -Sebastien >> >>> >>> >>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd < >> darren.s.sheph...@gmail.com> wrote: >>> >>>> After much searching I found the two lines of code I needed to get it >> to serialize right. I'll keep looking and see if I find other snags. >> Maybe we can just move to jackson.... we'll see, haven't tried to >> deserialize yet. >>>> >>>> Darren >>>> >>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> wrote: >>>>> >>>>> Darren, >>>>> >>>>> When I looked into updating to the latest gson, the problem, IIRC, is >> things that weren't considered cyclical dependencies are suddenly >> considered cyclical with the latest. I don't recall where though. >>>>> >>>>> --Alex >>>>> >>>>>> -----Original Message----- >>>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>>>> Sent: Thursday, September 19, 2013 1:33 PM >>>>>> To: dev@cloudstack.apache.org >>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>> >>>>>> Alright, I looked into this and it will take a bit more work to >> switch to Jackson. >>>>>> The snag I hit was how we serialize commands for the agents. We use a >>>>>> customer typeadatper in gson to produce a format like { "org.MyClass" >>>>>> : {...} }. In jackson I don't see producing a similar format as >> being so straight >>>>>> forward. I'm sure there's an elegant solution, but I didn't >> immediately find it. >>>>>> The problem is if you use a custom serializer in jackson and do >>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a >> loop >>>>>> because it will call your serializer again. So if somebody can >> figure out how to >>>>>> do this format easily in jackson, the rest looks simple enough. >>>>>> >>>>>> So, being that it is not an obvious switch to using jackson, what is >> the impact >>>>>> of moving to the latest gson? What were the issue encountered when >>>>>> people tried it before? >>>>>> >>>>>> Darren >>>>>> >>>>>> >>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>>>> darren.s.sheph...@gmail.com> wrote: >>>>>> >>>>>>> I can do some analysis on this. I'm always up for a terribly painful >>>>>>> refactor :) >>>>>>> >>>>>>> Darren >>>>>>> >>>>>>> >>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <alex.hu...@citrix.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I actually did a quick try to update cloudstack to use newest gson >>>>>>>> version about 3 months back. Had to roll it back but I didn't try >>>>>>>> very hard though. Part of the reason why I decided to rollback is >>>>>>>> due to gson is used differently by various components in CloudStack >>>>>>>> and I didn't have time to get into each component to see if I break >>>>>>>> anything. The other is also I thought using Jackson would be much >>>>>>>> better and thought our next attempt should be with Jackson. >>>>>>>> >>>>>>>> Not that any of these helps you. Just know updating CloudStack to >>>>>>>> latest gson is not simple. >>>>>>>> >>>>>>>> --Alex >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>>>> To: d...@whirr.apache.org >>>>>>>>> Cc: dev@cloudstack.apache.org >>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>>>> >>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just >>>>>>>>> add >>>>>>>> the >>>>>>>>> feature url. Bonus: you can also use jclouds shell interface (part >>>>>>>>> of >>>>>>>> jclouds cli). >>>>>>>>> >>>>>>>>> Another option is to upgrade the CloudStack API to use the new >> version. >>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>>>> >>>>>>>>>> Dear all, >>>>>>>>>> >>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>>>> conflicting >>>>>>>>> issue. >>>>>>>>>> >>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>>>> gson >>>>>>>> 1.7.1. >>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>>>> >>>>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>>>> com/google/gson/TypeAdapter >>>>>>>>>> >>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will >>>>>>>>>> not build successfully due to the following test error. >>>>>>>>>> >>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>>>> ------------------------------**------------------------------** >>>>>>>>>> ------------ >>>>>>>>>> [INFO] >>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>>>> cloud-core >>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>>>> [INFO] >>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) >>>>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8' >>>>>>>>>> encoding to copy filtered resources. >>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>>>> 3 >>>>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile >>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered >> resources. >>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>>>> 3 >>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source >>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO] >>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>>>> cloud-core >>>>>>>>>> --- >>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>>>> target/surefire-reports >>>>>>>>>> >>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>> T E S T S >>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>>>> log4j:WARN See >>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>>>> http://logging.apa >>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>>>> >>>>>>>>>> Results : >>>>>>>>>> >>>>>>>>>> Failed tests: >> testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>>>> >>>>>>>>>> Tests in error: >>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>>>> >>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>>>> API --launchCluster to CloudStack, the following error related to >>>>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>>>> >>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught >>>>>>>>>> problem with >>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy" >>>>>>>>>> :"** >>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>> **LibvirtComputingResource","* >>>>>>>>>> * >>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>>>> 10.244.18 >>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-* >>>>>>>>>> * >>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource >>>>>>>>>> ","** >>>>>>>>>> contextMap":{},"wait":0}}] >>>>>>>>>> java.lang.ClassCastException: >>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>>>> cannot be cast to java.lang.Class >>>>>>>>>> at >>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>>>> at >>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>>>> TreeTypeAdapter.java:58) >>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>>>> at >>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>>>> java:235) >>>>>>>>>> at >>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>>>> at >>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>>>> ) >>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>>>> following exception but pushing on >>>>>>>>>> >>>>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>>>> then quickly died. >>>>>>>>>> Running on provider cloudstack using identity >>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>>>> ZRGXVO1VApG >>>>>>>>>> 6q0a >>>>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:) >>>>>>>>>> Configuring template for >>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>>>> (729061754@qtp-385354117- >>>>>>>>> 0:) >>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>>>> jobtracker >>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) >>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>>>> >>>>>>>>>> I attached the debuging log to this email. >>>>>>>>>> >>>>>>>>>> The launchCluster API is simple, it calls the >>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>>>> >>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>>>> LaunchClusterCommand command = null; >>>>>>>>>> try { >>>>>>>>>> command = new LaunchClusterCommand(); >>>>>>>>>> } catch (Exception ex) { >>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>> VER >>>>>>>>>> E, >>>>>>>>>> null, ex); >>>>>>>>>> } >>>>>>>>>> String[] args = new String[2]; >>>>>>>>>> args[0] = "--config"; >>>>>>>>>> args[1] = config; >>>>>>>>>> >>>>>>>>>> try { >>>>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>>>> Arrays.asList(args)); >>>>>>>>>> } catch (Exception ex) { >>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>> VER >>>>>>>>>> E, >>>>>>>>>> null, ex); >>>>>>>>>> } >>>>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>>>> output = "successfully launched the cluster."; >>>>>>>>>> response.setOutPut(output); >>>>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>>>> this.setResponseObject(**response); >>>>>>>>>> >>>>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>>>> >>>>>>>>>> Thanks loads. >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> Meng >>>>>>> >>>>>>> >>> >> >>
