Alright, I looked into this and it will take a bit more work to switch to
Jackson. The snag I hit was how we serialize commands for the agents. We
use a customer typeadatper in gson to produce a format like { "org.MyClass"
: {...} }. In jackson I don't see producing a similar format as being so
straight forward. I'm sure there's an elegant solution, but I didn't
immediately find it. The problem is if you use a custom serializer in
jackson and do writeObjectField(x.getClass().getName(), x), you'll get
stuck in a loop because it will call your serializer again. So if somebody
can figure out how to do this format easily in jackson, the rest looks
simple enough.
So, being that it is not an obvious switch to using jackson, what is the
impact of moving to the latest gson? What were the issue encountered when
people tried it before?
Darren
On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
darren.s.sheph...@gmail.com> wrote:
> I can do some analysis on this. I'm always up for a terribly painful
> refactor :)
>
> Darren
>
>
> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <alex.hu...@citrix.com> wrote:
>
>> I actually did a quick try to update cloudstack to use newest gson
>> version about 3 months back. Had to roll it back but I didn't try very
>> hard though. Part of the reason why I decided to rollback is due to gson
>> is used differently by various components in CloudStack and I didn't have
>> time to get into each component to see if I break anything. The other is
>> also I thought using Jackson would be much better and thought our next
>> attempt should be with Jackson.
>>
>> Not that any of these helps you. Just know updating CloudStack to latest
>> gson is not simple.
>>
>> --Alex
>>
>> > -----Original Message-----
>> > From: Andrei Savu [mailto:savu.and...@gmail.com]
>> > Sent: Wednesday, September 18, 2013 10:53 AM
>> > To: d...@whirr.apache.org
>> > Cc: dev@cloudstack.apache.org
>> > Subject: Re: conflicting dependencies between CloudStack and Whirr
>> >
>> > It's easy to usr jclouds and whirr inside an OSGi container - just add
>> the
>> > feature url. Bonus: you can also use jclouds shell interface (part of
>> jclouds cli).
>> >
>> > Another option is to upgrade the CloudStack API to use the new version.
>> > On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote:
>> >
>> > > Dear all,
>> > >
>> > > I am adding an API to CloudStack which utilizes Whirr to launch
>> > > various clusters on CloudStack. Now I am facing a dependency
>> conflicting
>> > issue.
>> > >
>> > > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
>> 1.7.1.
>> > > If I use gson 1.7.1 for Whirr, the following error will happen:
>> > >
>> > > com.google.common.util.**concurrent.ExecutionError:
>> > java.lang.**NoClassDefFoundError:
>> > > com/google/gson/TypeAdapter
>> > >
>> > > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
>> > > I modify CloudStack to use gson 2.2.2 CloudStack will not build
>> > > successfully due to the following test error.
>> > >
>> > > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>> > > ------------------------------**------------------------------**
>> > > ------------
>> > > [INFO]
>> > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
>> > > --- [INFO] Deleting /home/meng/cloudstack/core/**target
>> > > [INFO]
>> > > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
>> > > cloud-core --- [INFO] [INFO] ---
>> > > maven-resources-plugin:2.5:**resources (default-resources) @
>> > > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>> > > encoding to copy filtered resources.
>> > > [INFO] skip non existing resourceDirectory
>> > > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
>> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>> > > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
>> > > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
>> > > maven-resources-plugin:2.5:**testResources
>> > > (default-testResources) @ cloud-core --- [debug] execute contextualize
>> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > > [INFO] skip non existing resourceDirectory
>> > > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
>> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
>> > > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
>> > > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
>> > > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
>> > > ---
>> > > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>> > > target/surefire-reports
>> > >
>> > > ------------------------------**-------------------------
>> > > T E S T S
>> > > ------------------------------**-------------------------
>> > > Running com.cloud.agent.transport.**RequestTest
>> > > log4j:WARN No appenders could be found for logger
>> > > (com.cloud.agent.transport.**RequestTest).
>> > > log4j:WARN Please initialize the log4j system properly.
>> > > log4j:WARN See
>> > http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>> http://logging.apa
>> > che.org/log4j/1.2/faq.html#noconfig>for more info.
>> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
>> > > sec <<< FAILURE!
>> > >
>> > > Results :
>> > >
>> > > Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>> > >
>> > > Tests in error:
>> > > testLogging(com.cloud.agent.**transport.RequestTest)
>> > >
>> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>> > >
>> > >
>> > > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
>> > > building process went through. But when I issued an API
>> > > --launchCluster to CloudStack, the following error related to gson
>> > > popped up on CloudStack Management Server:
>> > >
>> > > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>> > > problem with
>> > > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>> > > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>> > > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>> > > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>> > > snapshot","pool":"/root","**hypervisorType":"KVM","**
>> > >
>> > hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>> > > :"**
>> > > HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>> > > 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>> > > Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>> > > "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>> > **LibvirtComputingResource","*
>> > > *
>> > > name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>> > > publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>> > > 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>> > > privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>> > > 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>> > > storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>> > > 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>> > > LibvirtComputingResource","**gatewayIpAddress":"!
>> > > 10.244.18
>> > > .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>> > > totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>> > > 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>> > > lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>> > > images/","poolType":"**Filesystem","capacityBytes":**
>> > > 52844687360,"availableBytes":**41535332352},"resourceType":"**
>> > > STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>> > > :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
>> > > LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>> > > :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
>> > > contextMap":{},"wait":0}}]
>> > > java.lang.ClassCastException:
>> > > com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>> > > cannot be cast to java.lang.Class
>> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> > > ArrayTypeAdaptor.java:84)
>> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> > > ArrayTypeAdaptor.java:37)
>> > > at com.google.gson.**TreeTypeAdapter.read(**
>> > > TreeTypeAdapter.java:58)
>> > > at com.google.gson.Gson.fromJson(**Gson.java:795)
>> > > at com.cloud.agent.transport.**Request.getCommands(Request.**
>> > > java:235)
>> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> > > processRequest(**AgentManagerImpl.java:1221)
>> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> > > doTask(AgentManagerImpl.java:**1374)
>> > > at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>> > >
>> > ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
>> > > at com.cloud.utils.nio.Task.run(**Task.java:83)
>> > > at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>> > > ThreadPoolExecutor.java:1110)
>> > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> > > ThreadPoolExecutor.java:603)
>> > > at java.lang.Thread.run(Thread.**java:722)
>> > > WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
>> > > exception but pushing on
>> > >
>> > > On the CS Managment console I can see the cluster was started then
>> > > quickly died.
>> > > Running on provider cloudstack using identity
>> > >
>> > h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>> > ZRGXVO1VApG
>> > > 6q0a
>> > > **K-A-tQRQsZFwnOXQ
>> > > INFO [whirr.actions.**BootstrapClusterAction]
>> > > (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>> > > [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>> > > Configuring template for
>> > > bootstrap-hadoop-datanode_**hadoop-tasktracker
>> > > INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-
>> > 0:)
>> > > Configuring template for bootstrap-hadoop-namenode_**hadoop-
>> > jobtracker
>> > > INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>> > > node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>> > > [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) with
>> > > roles [hadoop-namenode, hadoop-jobtracker] ...
>> > > INFO [whirr.actions.**DestroyClusterAction]
>> > > (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>> > >
>> > > I attached the debuging log to this email.
>> > >
>> > > The launchCluster API is simple, it calls the LaunchClusterCommand in
>> > > Whirr to launch a cluster.
>> > >
>> > > LaunchClusterResponse response = new LaunchClusterResponse();
>> > > response.setObjectName("**launchCluster");
>> > > LaunchClusterCommand command = null;
>> > > try {
>> > > command = new LaunchClusterCommand();
>> > > } catch (Exception ex) {
>> > >
>> > >
>> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
>> > > E,
>> > > null, ex);
>> > > }
>> > > String[] args = new String[2];
>> > > args[0] = "--config";
>> > > args[1] = config;
>> > >
>> > > try {
>> > > command.run(System.in, System.out, System.err,
>> > > Arrays.asList(args));
>> > > } catch (Exception ex) {
>> > >
>> > >
>> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
>> > > E,
>> > > null, ex);
>> > > }
>> > > response.setResponseName(**getCommandName());
>> > > output = "successfully launched the cluster.";
>> > > response.setOutPut(output);
>> > > response.setAsync(Boolean.**FALSE);
>> > > this.setResponseObject(**response);
>> > >
>> > > Could someone help me out here? What would be a proper gson version
>> > > for CloudStack and Whirr to run at the same time?
>> > >
>> > > Thanks loads.
>> > >
>> > > Best Regards,
>> > > Meng
>> > >
>> > >
>> > >
>>
>
>
