This is what a fresh agent.properties file looks like on my system. I expect if I try to add it to a cluster, the empty, localhost, and default values below should be filled in.
I plan to try to add it to a cluster in a bit. # The GUID to identify the agent with, this is mandatory! # Generate with "uuidgen" guid= #resource= the java class, which agent load to execute resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource #workers= number of threads running in agent workers=5 #host= The IP address of management server host=localhost #port = The port management server listening on, default is 8250 port=8250 #cluster= The cluster which the agent belongs to cluster=default #pod= The pod which the agent belongs to pod=default #zone= The zone which the agent belongs to zone=default On Tue, Sep 24, 2013 at 8:55 AM, Marcus Sorensen <shadow...@gmail.com>wrote: > I stull haven't seen your agent.properties. This would tell me if your > setup succeeded. At this point my best guess is that > "cloudstack-setup-agent -m 192.168.233.1 -z 1 -p 1 -c 1 -g > 6b4aa1c2-2ac9-3c60-aabe-704aed40c684 -a --pubNic=cloudbr0 > --prvNic=cloudbr0 --guestNic=cloudbr0" failed in some fashion. You can > run it manually at any time to see. Once that is run, then the agent > should come up. The resource name in your code is pulled from > agent.properties (I believe) and is usually > "resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource". > > On Tue, Sep 24, 2013 at 12:12 AM, Mike Tutkowski > <mike.tutkow...@solidfire.com> wrote: > > I've been narrowing it down by putting in a million print-to-log > statements. > > > > Do you know if it is a problem that value ends up null (in a constructor > > for Agent)? > > > > String value = _shell.getPersistentProperty(getResourceName(), "id"); > > > > In that same constructor, this line never finishes: > > > > if (!_resource.configure(getResourceName(), params)) { > > > > I need to dig into the configure method to see what's going on there. > > > > > > On Mon, Sep 23, 2013 at 5:45 PM, Marcus Sorensen <shadow...@gmail.com > >wrote: > > > >> It might be a centos specific thing. These are created by the init > scripts. > >> Check your agent init script on Ubuntu and see I'd you can decipher > where > >> it sends stdout. > >> On Sep 23, 2013 5:21 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com > > > >> wrote: > >> > >> > Weird...no such file exists. > >> > > >> > > >> > On Mon, Sep 23, 2013 at 4:54 PM, Marcus Sorensen <shadow...@gmail.com > >> > >wrote: > >> > > >> > > maybe cloudstack-agent.out > >> > > > >> > > On Mon, Sep 23, 2013 at 4:44 PM, Mike Tutkowski > >> > > <mike.tutkow...@solidfire.com> wrote: > >> > > > OK, so, nothing is screaming out in the logs. I did notice the > >> > following: > >> > > > > >> > > > From setup.log: > >> > > > > >> > > > DEBUG:root:execute:apparmor_status |grep libvirt > >> > > > > >> > > > DEBUG:root:Failed to execute: > >> > > > > >> > > > > >> > > > DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent status > >> > > > > >> > > > DEBUG:root:Failed to execute: * could not access PID file for > >> > > > cloudstack-agent > >> > > > > >> > > > > >> > > > This is the final line in this log file: > >> > > > > >> > > > DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent start > >> > > > > >> > > > > >> > > > This is from agent.log: > >> > > > > >> > > > 2013-09-23 15:30:55,549 DEBUG [cloud.agent.AgentShell] (main:null) > >> > > Checking > >> > > > to see if agent.pid exists. > >> > > > > >> > > > 2013-09-23 15:30:55,655 DEBUG [cloud.utils.ProcessUtil] > (main:null) > >> > > > Executing: bash -c echo $PPID > >> > > > > >> > > > 2013-09-23 15:30:55,742 DEBUG [cloud.utils.ProcessUtil] > (main:null) > >> > > > Execution is successful. > >> > > > > >> > > > 2013-09-23 15:30:56,000 INFO [cloud.agent.Agent] (main:null) id > is > >> > > > > >> > > > 2013-09-23 15:30:56,000 DEBUG [cloud.resource.ServerResourceBase] > >> > > > (main:null) Retrieving network interface: cloudbr0 > >> > > > > >> > > > 2013-09-23 15:30:56,016 DEBUG [cloud.resource.ServerResourceBase] > >> > > > (main:null) Retrieving network interface: cloudbr0 > >> > > > > >> > > > 2013-09-23 15:30:56,016 DEBUG [cloud.resource.ServerResourceBase] > >> > > > (main:null) Retrieving network interface: null > >> > > > > >> > > > 2013-09-23 15:30:56,017 DEBUG [cloud.resource.ServerResourceBase] > >> > > > (main:null) Retrieving network interface: null > >> > > > > >> > > > > >> > > > The following kinds of lines are repeated for a bunch of different > >> .sh > >> > > > files. I think they often end up being found here: > >> > > > /usr/share/cloudstack-common/scripts/network/domr, so this is > >> probably > >> > > not > >> > > > an issue. > >> > > > > >> > > > > >> > > > 2013-09-23 15:30:56,111 DEBUG [utils.script.Script] (main:null) > >> Looking > >> > > for > >> > > > call_firewall.sh in the classpath > >> > > > > >> > > > 2013-09-23 15:30:56,112 DEBUG [utils.script.Script] (main:null) > >> System > >> > > > resource: null > >> > > > > >> > > > 2013-09-23 15:30:56,113 DEBUG [utils.script.Script] (main:null) > >> > Classpath > >> > > > resource: null > >> > > > > >> > > > 2013-09-23 15:30:56,123 DEBUG [utils.script.Script] (main:null) > >> Looking > >> > > for > >> > > > call_firewall.sh > >> > > > > >> > > > > >> > > > Is there a log file for the Java code that I could write stuff > out to > >> > and > >> > > > see how far we get? > >> > > > > >> > > > > >> > > > On Mon, Sep 23, 2013 at 3:17 PM, Mike Tutkowski < > >> > > > mike.tutkow...@solidfire.com> wrote: > >> > > > > >> > > >> Thanks, Marcus > >> > > >> > >> > > >> I've been developing on Windows for most of my time, so a bunch > of > >> > these > >> > > >> Linux-type commands are new to me and I don't always interpret > the > >> > > output > >> > > >> correctly. Getting there. :) > >> > > >> > >> > > >> > >> > > >> On Mon, Sep 23, 2013 at 2:37 PM, Marcus Sorensen < > >> shadow...@gmail.com > >> > > >wrote: > >> > > >> > >> > > >>> Nope, not running. That's just your grep process. It would look > >> like: > >> > > >>> > >> > > >>> root 24429 24428 1 14:25 ? 00:00:08 jsvc.exec -cp > >> > > >>> > >> > > >>> > >> > > > >> > > >> > /usr/share/java/commons-daemon.jar:/usr/share/java/jna.jar:/usr/share/cloudstack-agent/lib/activation-1.1.jar:/usr/share/cloudstack-agent/lib/antisamy-1.4.3.jar:/usr/share/cloudstack-agent/lib/aopalliance-1.0.jar:/usr/share/cloudstack-agent/lib/apache-log4j-extras-1.1.jar:/usr/share/cloudstack-agent/lib/aspectjrt-1.7. > >> > > >>> > >> > > >>> Your agent log should tell you why it failed to start if you > set it > >> > in > >> > > >>> debug and try to start... or maybe cloudstack-agent.out if it > >> doesn't > >> > > >>> get far enough (say it's missing a class or something and can't > >> > > >>> start). > >> > > >>> > >> > > >>> On Mon, Sep 23, 2013 at 2:33 PM, Mike Tutkowski > >> > > >>> <mike.tutkow...@solidfire.com> wrote: > >> > > >>> > Looks like it's running, though: > >> > > >>> > > >> > > >>> > mtutkowski@ubuntu:~$ ps -ef | grep jsvc > >> > > >>> > 1000 7097 7013 0 14:32 pts/1 00:00:00 grep > --color=auto > >> > > jsvc > >> > > >>> > > >> > > >>> > > >> > > >>> > > >> > > >>> > On Mon, Sep 23, 2013 at 2:31 PM, Mike Tutkowski < > >> > > >>> > mike.tutkow...@solidfire.com> wrote: > >> > > >>> > > >> > > >>> >> Hey Marcus, > >> > > >>> >> > >> > > >>> >> Maybe you could give me a better idea of what the "flow" is > when > >> > > >>> adding a > >> > > >>> >> KVM host. > >> > > >>> >> > >> > > >>> >> It looks like we SSH into the potential KVM host and execute > a > >> > > startup > >> > > >>> >> script (giving it necessary info about the cloud and the > >> > management > >> > > >>> server > >> > > >>> >> it should talk to). > >> > > >>> >> > >> > > >>> >> After this, is the Java VM started? > >> > > >>> >> > >> > > >>> >> After a reboot, I assume the JVM is started automatically? > >> > > >>> >> > >> > > >>> >> How do you debug your KVM-side Java code? > >> > > >>> >> > >> > > >>> >> Been looking through the logs and nothing obvious sticks > out. I > >> > will > >> > > >>> have > >> > > >>> >> another look. > >> > > >>> >> > >> > > >>> >> Thanks > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> On Mon, Sep 23, 2013 at 2:15 PM, Mike Tutkowski < > >> > > >>> >> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >> > >> > > >>> >>> Hey Marcus, > >> > > >>> >>> > >> > > >>> >>> I've been investigating my issue with not being able to add > a > >> KVM > >> > > >>> host to > >> > > >>> >>> CS. > >> > > >>> >>> > >> > > >>> >>> For what it's worth, this comes back successful: > >> > > >>> >>> > >> > > >>> >>> SSHCmdHelper.sshExecuteCmd(sshConnection, > >> "cloudstack-setup-agent > >> > > " + > >> > > >>> >>> parameters, 3); > >> > > >>> >>> > >> > > >>> >>> This is what the command looks like: > >> > > >>> >>> > >> > > >>> >>> cloudstack-setup-agent -m 192.168.233.1 -z 1 -p 1 -c 1 -g > >> > > >>> >>> 6b4aa1c2-2ac9-3c60-aabe-704aed40c684 -a --pubNic=cloudbr0 > >> > > >>> --prvNic=cloudbr0 > >> > > >>> >>> --guestNic=cloudbr0 > >> > > >>> >>> > >> > > >>> >>> The problem is this method in LibvirtServerDiscoverer never > >> > finds a > >> > > >>> >>> matching host in the DB: > >> > > >>> >>> > >> > > >>> >>> waitForHostConnect(long dcId, long podId, long clusterId, > >> String > >> > > guid) > >> > > >>> >>> > >> > > >>> >>> I assume once the KVM host is up and running that it's > supposed > >> > to > >> > > >>> call > >> > > >>> >>> into the CS MS so the DB can be updated as such? > >> > > >>> >>> > >> > > >>> >>> If so, the problem must be on the KVM side. > >> > > >>> >>> > >> > > >>> >>> I did run this again (from the KVM host) to see if the > >> connection > >> > > was > >> > > >>> in > >> > > >>> >>> place: > >> > > >>> >>> > >> > > >>> >>> mtutkowski@ubuntu:~$ telnet 192.168.233.1 8250 > >> > > >>> >>> > >> > > >>> >>> Trying 192.168.233.1... > >> > > >>> >>> > >> > > >>> >>> Connected to 192.168.233.1. > >> > > >>> >>> > >> > > >>> >>> Escape character is '^]'. > >> > > >>> >>> So that looks good. > >> > > >>> >>> > >> > > >>> >>> I turned on more info in the debug log, but nothing obvious > >> jumps > >> > > out > >> > > >>> as > >> > > >>> >>> of yet. > >> > > >>> >>> > >> > > >>> >>> If you have any thoughts on this, please shoot them my way. > :) > >> > > >>> >>> > >> > > >>> >>> Thanks! > >> > > >>> >>> > >> > > >>> >>> > >> > > >>> >>> On Sun, Sep 22, 2013 at 12:11 AM, Mike Tutkowski < > >> > > >>> >>> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>> > >> > > >>> >>>> First step is for me to get this working for KVM, though. > :) > >> > > >>> >>>> > >> > > >>> >>>> Once I do that, I can perhaps make modifications to the > >> storage > >> > > >>> >>>> framework and hypervisor plug-ins to refactor the logic and > >> > such. > >> > > >>> >>>> > >> > > >>> >>>> > >> > > >>> >>>> On Sun, Sep 22, 2013 at 12:09 AM, Mike Tutkowski < > >> > > >>> >>>> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>> > >> > > >>> >>>>> Same would work for KVM. > >> > > >>> >>>>> > >> > > >>> >>>>> If CreateCommand and DestroyCommand were called at the > >> > > appropriate > >> > > >>> >>>>> times by the storage framework, I could move my connect > and > >> > > >>> disconnect > >> > > >>> >>>>> logic out of the attach/detach logic. > >> > > >>> >>>>> > >> > > >>> >>>>> > >> > > >>> >>>>> On Sun, Sep 22, 2013 at 12:08 AM, Mike Tutkowski < > >> > > >>> >>>>> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>> > >> > > >>> >>>>>> Conversely, if the storage framework called the > >> DestroyCommand > >> > > for > >> > > >>> >>>>>> managed storage after the DetachCommand, then I could > have > >> had > >> > > my > >> > > >>> remove > >> > > >>> >>>>>> SR/datastore logic placed in the DestroyCommand handling > >> > rather > >> > > >>> than in the > >> > > >>> >>>>>> DetachCommand handling. > >> > > >>> >>>>>> > >> > > >>> >>>>>> > >> > > >>> >>>>>> On Sun, Sep 22, 2013 at 12:06 AM, Mike Tutkowski < > >> > > >>> >>>>>> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>> > >> > > >>> >>>>>>> Edison's plug-in calls the CreateCommand. Mine does not. > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> The initial approach that was discussed during 4.2 was > for > >> me > >> > > to > >> > > >>> >>>>>>> modify the attach/detach logic only in the XenServer and > >> > VMware > >> > > >>> hypervisor > >> > > >>> >>>>>>> plug-ins. > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> Now that I think about it more, though, I kind of would > >> have > >> > > >>> liked to > >> > > >>> >>>>>>> have the storage framework send a CreateCommand to the > >> > > hypervisor > >> > > >>> before > >> > > >>> >>>>>>> sending the AttachCommand if the storage in question was > >> > > managed. > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> Then I could have created my SR/datastore in the > >> > CreateCommand > >> > > and > >> > > >>> >>>>>>> the AttachCommand would have had the SR/datastore that > it > >> was > >> > > >>> always > >> > > >>> >>>>>>> expecting (and I wouldn't have had to create the > >> SR/datastore > >> > > in > >> > > >>> the > >> > > >>> >>>>>>> AttachCommand). > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> On Sat, Sep 21, 2013 at 11:56 PM, Marcus Sorensen < > >> > > >>> >>>>>>> shadow...@gmail.com> wrote: > >> > > >>> >>>>>>> > >> > > >>> >>>>>>>> Yeah, I think it probably is as well, but I figured > you'd > >> be > >> > > in a > >> > > >>> >>>>>>>> better position to tell. > >> > > >>> >>>>>>>> > >> > > >>> >>>>>>>> I see that copyAsync is unsupported in your current 4.2 > >> > > driver, > >> > > >>> does > >> > > >>> >>>>>>>> that mean that there's no template support? Or is it > some > >> > > other > >> > > >>> call > >> > > >>> >>>>>>>> that does templating now? I'm still getting up to > speed on > >> > all > >> > > >>> of the > >> > > >>> >>>>>>>> 4.2 changes. I was just looking at CreateCommand in > >> > > >>> >>>>>>>> LibvirtComputingResource, since that's the only place > >> > > >>> >>>>>>>> createPhysicalDisk is called, and it occurred to me > that > >> > > >>> >>>>>>>> CreateCommand > >> > > >>> >>>>>>>> might be skipped altogether when utilizing storage > >> plugins. > >> > > >>> >>>>>>>> > >> > > >>> >>>>>>>> On Sat, Sep 21, 2013 at 11:38 PM, Mike Tutkowski > >> > > >>> >>>>>>>> <mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>>>> > That's an interesting comment, Marcus. > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > It was my intent that it should work with any > CloudStack > >> > > >>> "managed" > >> > > >>> >>>>>>>> storage > >> > > >>> >>>>>>>> > that uses an iSCSI target. Even though I'm using > CHAP, I > >> > > wrote > >> > > >>> the > >> > > >>> >>>>>>>> code so > >> > > >>> >>>>>>>> > CHAP didn't have to be used. > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > As I'm doing my testing, I can try to think about > >> whether > >> > > it is > >> > > >>> >>>>>>>> generic > >> > > >>> >>>>>>>> > enough to keep those names or not. > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > My expectation is that it is generic enough. > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen < > >> > > >>> >>>>>>>> shadow...@gmail.com>wrote: > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> >> I added a comment to your diff. In general I think > it > >> > looks > >> > > >>> good, > >> > > >>> >>>>>>>> >> though I obviously can't vouch for whether or not it > >> will > >> > > >>> work. > >> > > >>> >>>>>>>> One > >> > > >>> >>>>>>>> >> thing I do have reservations about is the > adaptor/pool > >> > > >>> naming. If > >> > > >>> >>>>>>>> you > >> > > >>> >>>>>>>> >> think the code is generic enough that it will work > for > >> > > anyone > >> > > >>> who > >> > > >>> >>>>>>>> does > >> > > >>> >>>>>>>> >> an iscsi LUN-per-volume plugin, then it's OK, but if > >> > > there's > >> > > >>> >>>>>>>> anything > >> > > >>> >>>>>>>> >> about it that's specific to YOUR iscsi target or > how it > >> > > likes > >> > > >>> to > >> > > >>> >>>>>>>> be > >> > > >>> >>>>>>>> >> treated then I'd say that they should be named > >> something > >> > > less > >> > > >>> >>>>>>>> generic > >> > > >>> >>>>>>>> >> than iScsiAdmStorage. > >> > > >>> >>>>>>>> >> > >> > > >>> >>>>>>>> >> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski > >> > > >>> >>>>>>>> >> <mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>>>> >> > Great - thanks! > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > Just to give you an overview of what my code does > >> (for > >> > > when > >> > > >>> you > >> > > >>> >>>>>>>> get a > >> > > >>> >>>>>>>> >> > chance to review it): > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > SolidFireHostListener is registered in > >> > > >>> >>>>>>>> SolidfirePrimaryDataStoreProvider. > >> > > >>> >>>>>>>> >> > Its hostConnect method is invoked when a host > >> connects > >> > > with > >> > > >>> the > >> > > >>> >>>>>>>> CS MS. If > >> > > >>> >>>>>>>> >> > the host is running KVM, the listener sends a > >> > > >>> >>>>>>>> ModifyStoragePoolCommand to > >> > > >>> >>>>>>>> >> > the host. This logic was based off of > >> > > DefaultHostListener. > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > The handling of ModifyStoragePoolCommand is > >> unchanged. > >> > It > >> > > >>> >>>>>>>> invokes > >> > > >>> >>>>>>>> >> > createStoragePool on the KVMStoragePoolManager. > The > >> > > >>> >>>>>>>> KVMStoragePoolManager > >> > > >>> >>>>>>>> >> > asks for an adaptor and finds my new one: > >> > > >>> >>>>>>>> iScsiAdmStorageAdaptor (which > >> > > >>> >>>>>>>> >> was > >> > > >>> >>>>>>>> >> > registered in the constructor for > >> KVMStoragePoolManager > >> > > >>> under > >> > > >>> >>>>>>>> the key of > >> > > >>> >>>>>>>> >> > StoragePoolType.Iscsi.toString()). > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > iScsiAdmStorageAdaptor.createStoragePool just > makes > >> an > >> > > >>> instance > >> > > >>> >>>>>>>> of > >> > > >>> >>>>>>>> >> > iScsiAdmStoragePool, adds it to a map, and returns > >> the > >> > > >>> pointer > >> > > >>> >>>>>>>> to the > >> > > >>> >>>>>>>> >> > iScsiAdmStoragePool object. The key of the map is > the > >> > > UUID > >> > > >>> of > >> > > >>> >>>>>>>> the storage > >> > > >>> >>>>>>>> >> > pool. > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > When a volume is attached, createPhysicalDisk is > >> > invoked > >> > > for > >> > > >>> >>>>>>>> managed > >> > > >>> >>>>>>>> >> > storage rather than getPhysicalDisk. > >> createPhysicalDisk > >> > > uses > >> > > >>> >>>>>>>> iscsiadm to > >> > > >>> >>>>>>>> >> > establish the iSCSI connection to the volume on > the > >> SAN > >> > > and > >> > > >>> a > >> > > >>> >>>>>>>> >> > KVMPhysicalDisk is returned to be used in the > attach > >> > > logic > >> > > >>> that > >> > > >>> >>>>>>>> follows. > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > When a volume is detached, getPhysicalDisk is > invoked > >> > > with > >> > > >>> the > >> > > >>> >>>>>>>> IQN of the > >> > > >>> >>>>>>>> >> > volume if the storage pool in question is managed > >> > > storage. > >> > > >>> >>>>>>>> Otherwise, the > >> > > >>> >>>>>>>> >> > normal vol.getPath() is used. > >> > > >>> >>>>>>>> iScsiAdmStorageAdaptor.getPhysicalDisk just > >> > > >>> >>>>>>>> >> > returns a new instance of KVMPhysicalDisk to be > used > >> in > >> > > the > >> > > >>> >>>>>>>> detach logic. > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > Once the volume has been detached, > >> > > >>> >>>>>>>> iScsiAdmStoragePool.deletePhysicalDisk > >> > > >>> >>>>>>>> >> > is invoked if the storage pool is managed. > >> > > >>> deletePhysicalDisk > >> > > >>> >>>>>>>> removes the > >> > > >>> >>>>>>>> >> > iSCSI connection to the volume using iscsiadm. > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen < > >> > > >>> >>>>>>>> shadow...@gmail.com > >> > > >>> >>>>>>>> >> >wrote: > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> >> Its the log4j properties file in > >> /etc/cloudstack/agent > >> > > >>> change > >> > > >>> >>>>>>>> all INFO > >> > > >>> >>>>>>>> >> to > >> > > >>> >>>>>>>> >> >> DEBUG. I imagine the agent just isn't starting, > you > >> > can > >> > > >>> tail > >> > > >>> >>>>>>>> the log > >> > > >>> >>>>>>>> >> when > >> > > >>> >>>>>>>> >> >> you try to start the service, or maybe it will > spit > >> > > >>> something > >> > > >>> >>>>>>>> out into > >> > > >>> >>>>>>>> >> one > >> > > >>> >>>>>>>> >> >> of the other files in /var/log/cloudstack/agent > >> > > >>> >>>>>>>> >> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" < > >> > > >>> >>>>>>>> mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> >> wrote: > >> > > >>> >>>>>>>> >> >> > >> > > >>> >>>>>>>> >> >> > This is how I've been trying to query for the > >> status > >> > > of > >> > > >>> the > >> > > >>> >>>>>>>> service (I > >> > > >>> >>>>>>>> >> >> > assume it could be started this way, as well, > by > >> > > changing > >> > > >>> >>>>>>>> "status" to > >> > > >>> >>>>>>>> >> >> > "start" or "restart"?): > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo > >> > > >>> >>>>>>>> /usr/sbin/service > >> > > >>> >>>>>>>> >> >> > cloudstack-agent status > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > I get this back: > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > Failed to execute: * could not access PID file > for > >> > > >>> >>>>>>>> cloudstack-agent > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > I've made a bunch of code changes recently, > >> though, > >> > > so I > >> > > >>> >>>>>>>> think I'm > >> > > >>> >>>>>>>> >> going > >> > > >>> >>>>>>>> >> >> to > >> > > >>> >>>>>>>> >> >> > rebuild and redeploy everything. > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > The debug info sounds helpful. Where can I set > >> > > >>> enable.debug? > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > Thanks, Marcus! > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus > Sorensen < > >> > > >>> >>>>>>>> shadow...@gmail.com > >> > > >>> >>>>>>>> >> >> > >wrote: > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > > OK, will check it out in the next few days. > As > >> > > >>> mentioned, > >> > > >>> >>>>>>>> you can > >> > > >>> >>>>>>>> >> set > >> > > >>> >>>>>>>> >> >> up > >> > > >>> >>>>>>>> >> >> > > your Ubuntu vm as the management server as > well > >> if > >> > > all > >> > > >>> >>>>>>>> else fails. > >> > > >>> >>>>>>>> >> If > >> > > >>> >>>>>>>> >> >> > you > >> > > >>> >>>>>>>> >> >> > > can get to the mgmt server on 8250 from the > KVM > >> > > host, > >> > > >>> then > >> > > >>> >>>>>>>> you need > >> > > >>> >>>>>>>> >> to > >> > > >>> >>>>>>>> >> >> > > enable.debug on the agent. It won't run > without > >> > > >>> >>>>>>>> complaining loudly > >> > > >>> >>>>>>>> >> if > >> > > >>> >>>>>>>> >> >> it > >> > > >>> >>>>>>>> >> >> > > can't get to the mgmt server, and I didn't > see > >> > that > >> > > in > >> > > >>> >>>>>>>> your agent > >> > > >>> >>>>>>>> >> log, > >> > > >>> >>>>>>>> >> >> so > >> > > >>> >>>>>>>> >> >> > > perhaps its not running. I assume you know > how > >> to > >> > > >>> >>>>>>>> stop/start the > >> > > >>> >>>>>>>> >> agent > >> > > >>> >>>>>>>> >> >> on > >> > > >>> >>>>>>>> >> >> > > KVM via 'service cloud stacks agent'. > >> > > >>> >>>>>>>> >> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" < > >> > > >>> >>>>>>>> >> >> mike.tutkow...@solidfire.com> > >> > > >>> >>>>>>>> >> >> > > wrote: > >> > > >>> >>>>>>>> >> >> > > > >> > > >>> >>>>>>>> >> >> > > > Hey Marcus, > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > I haven't yet been able to test my new > code, > >> > but I > >> > > >>> >>>>>>>> thought you > >> > > >>> >>>>>>>> >> would > >> > > >>> >>>>>>>> >> >> > be a > >> > > >>> >>>>>>>> >> >> > > > good person to ask to review it: > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > >> > > >>> >>>>>>>> >> > >> > > >>> >>>>>>>> > >> > > >>> > >> > > > >> > > >> > https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37 > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > All it is supposed to do is attach and > detach > >> a > >> > > data > >> > > >>> >>>>>>>> disk (that > >> > > >>> >>>>>>>> >> has > >> > > >>> >>>>>>>> >> >> > > > guaranteed IOPS) with KVM as the > hypervisor. > >> The > >> > > data > >> > > >>> >>>>>>>> disk > >> > > >>> >>>>>>>> >> happens to > >> > > >>> >>>>>>>> >> >> > be > >> > > >>> >>>>>>>> >> >> > > > from SolidFire-backed storage - where we > have > >> a > >> > > 1:1 > >> > > >>> >>>>>>>> mapping > >> > > >>> >>>>>>>> >> between a > >> > > >>> >>>>>>>> >> >> > > > CloudStack volume and a data disk. > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > There is no support for hypervisor > snapshots > >> or > >> > > stuff > >> > > >>> >>>>>>>> like that > >> > > >>> >>>>>>>> >> >> > (likely a > >> > > >>> >>>>>>>> >> >> > > > future release)...just attaching and > >> detaching a > >> > > data > >> > > >>> >>>>>>>> disk in 4.3. > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > Thanks! > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike > >> > Tutkowski < > >> > > >>> >>>>>>>> >> >> > > > mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > When I re-deployed the DEBs, I didn't > remove > >> > > >>> >>>>>>>> cloudstack-agent > >> > > >>> >>>>>>>> >> >> first. > >> > > >>> >>>>>>>> >> >> > > > Would > >> > > >>> >>>>>>>> >> >> > > > > that be a problem? I just did a sudo > apt-get > >> > > >>> install > >> > > >>> >>>>>>>> >> >> > cloudstack-agent. > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike > >> > > Tutkowski < > >> > > >>> >>>>>>>> >> >> > > > > mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > >> I get the same error running the command > >> > > manually: > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> mtutkowski@ubuntu > :/etc/cloudstack/agent$ > >> > sudo > >> > > >>> >>>>>>>> >> /usr/sbin/service > >> > > >>> >>>>>>>> >> >> > > > >> cloudstack-agent status > >> > > >>> >>>>>>>> >> >> > > > >> * could not access PID file for > >> > > cloudstack-agent > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike > >> > > Tutkowski < > >> > > >>> >>>>>>>> >> >> > > > >> mike.tutkow...@solidfire.com> wrote: > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >>> agent.log looks OK to me: > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,010 INFO > >> > > >>> >>>>>>>> [cloud.agent.AgentShell] > >> > > >>> >>>>>>>> >> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > > Agent > >> > > >>> >>>>>>>> >> >> > > > >>> started > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,011 INFO > >> > > >>> >>>>>>>> [cloud.agent.AgentShell] > >> > > >>> >>>>>>>> >> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > > >>> Implementation Version is > 4.3.0-SNAPSHOT > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,015 INFO > >> > > >>> >>>>>>>> [cloud.agent.AgentShell] > >> > > >>> >>>>>>>> >> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > > >>> agent.properties found at > >> > > >>> >>>>>>>> >> /etc/cloudstack/agent/agent.properties > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,023 INFO > >> > > >>> >>>>>>>> [cloud.agent.AgentShell] > >> > > >>> >>>>>>>> >> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > > >>> Defaulting to using properties file for > >> > > storage > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,024 INFO > >> > > >>> >>>>>>>> [cloud.agent.AgentShell] > >> > > >>> >>>>>>>> >> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > > >>> Defaulting to the constant time backoff > >> > > algorithm > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,029 INFO > >> > > >>> [cloud.utils.LogUtils] > >> > > >>> >>>>>>>> >> (main:null) > >> > > >>> >>>>>>>> >> >> > > log4j > >> > > >>> >>>>>>>> >> >> > > > >>> configuration found at > >> > > >>> >>>>>>>> /etc/cloudstack/agent/log4j-cloud.xml > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,163 INFO > >> > > [cloud.agent.Agent] > >> > > >>> >>>>>>>> (main:null) > >> > > >>> >>>>>>>> >> id > >> > > >>> >>>>>>>> >> >> > is 3 > >> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,197 INFO > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > [resource.virtualnetwork.VirtualRoutingResource] > >> > > >>> >>>>>>>> (main:null) > >> > > >>> >>>>>>>> >> >> > > > >>> VirtualRoutingResource _scriptDir to > use: > >> > > >>> >>>>>>>> >> >> scripts/network/domr/kvm > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> However, I wasn't aware that setup.log > was > >> > > >>> >>>>>>>> important. This > >> > > >>> >>>>>>>> >> seems > >> > > >>> >>>>>>>> >> >> to > >> > > >>> >>>>>>>> >> >> > > be > >> > > >>> >>>>>>>> >> >> > > > a > >> > > >>> >>>>>>>> >> >> > > > >>> problem, but I'm not sure what it might > >> > > indicate: > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo > /usr/sbin/service > >> > > >>> >>>>>>>> cloudstack-agent > >> > > >>> >>>>>>>> >> status > >> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:Failed to execute: * could > not > >> > > access > >> > > >>> PID > >> > > >>> >>>>>>>> file for > >> > > >>> >>>>>>>> >> >> > > > >>> cloudstack-agent > >> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo > /usr/sbin/service > >> > > >>> >>>>>>>> cloudstack-agent > >> > > >>> >>>>>>>> >> start > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, > Marcus > >> > > >>> Sorensen < > >> > > >>> >>>>>>>> >> >> > > shadow...@gmail.com > >> > > >>> >>>>>>>> >> >> > > > >wrote: > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>>> Sorry, I saw that in the log, I > thought > >> it > >> > > was > >> > > >>> the > >> > > >>> >>>>>>>> agent log > >> > > >>> >>>>>>>> >> for > >> > > >>> >>>>>>>> >> >> > > some > >> > > >>> >>>>>>>> >> >> > > > >>>> reason. Is the agent started? That > might > >> be > >> > > the > >> > > >>> >>>>>>>> place to > >> > > >>> >>>>>>>> >> look. > >> > > >>> >>>>>>>> >> >> > There > >> > > >>> >>>>>>>> >> >> > > > is > >> > > >>> >>>>>>>> >> >> > > > >>>> an > >> > > >>> >>>>>>>> >> >> > > > >>>> agent log for the agent and one for > the > >> > setup > >> > > >>> when > >> > > >>> >>>>>>>> it adds > >> > > >>> >>>>>>>> >> the > >> > > >>> >>>>>>>> >> >> > host, > >> > > >>> >>>>>>>> >> >> > > > >>>> both > >> > > >>> >>>>>>>> >> >> > > > >>>> in /var/log > >> > > >>> >>>>>>>> >> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike > >> Tutkowski" > >> > < > >> > > >>> >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com> > >> > > >>> >>>>>>>> >> >> > > > >>>> wrote: > >> > > >>> >>>>>>>> >> >> > > > >>>> > >> > > >>> >>>>>>>> >> >> > > > >>>> > Is it saying that the MS is at the > IP > >> > > address > >> > > >>> or > >> > > >>> >>>>>>>> the KVM > >> > > >>> >>>>>>>> >> host? > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > The KVM host is at 192.168.233.10. > >> > > >>> >>>>>>>> >> >> > > > >>>> > The MS host is at 192.168.233.1. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > I see this for my host Global > Settings > >> > > >>> parameter: > >> > > >>> >>>>>>>> >> >> > > > >>>> > hostThe ip address of management > >> > > >>> >>>>>>>> server192.168.233.1 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > /etc/cloudstack/agent/agent.properties > >> > has > >> > > a > >> > > >>> >>>>>>>> >> >> host=192.168.233.1 > >> > > >>> >>>>>>>> >> >> > > > value. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM, > >> Marcus > >> > > >>> Sorensen > >> > > >>> >>>>>>>> < > >> > > >>> >>>>>>>> >> >> > > > >>>> shadow...@gmail.com > >> > > >>> >>>>>>>> >> >> > > > >>>> > >wrote: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > The log says your mgmt server is > >> > > >>> >>>>>>>> 192.168.233.10? But you > >> > > >>> >>>>>>>> >> >> tried > >> > > >>> >>>>>>>> >> >> > > to > >> > > >>> >>>>>>>> >> >> > > > >>>> telnet > >> > > >>> >>>>>>>> >> >> > > > >>>> > to > >> > > >>> >>>>>>>> >> >> > > > >>>> > > 192.168.233.1? It might be enough > to > >> > > change > >> > > >>> >>>>>>>> that in > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > >> /etc/cloudstack/agent/agent.properties, > >> > > but > >> > > >>> you > >> > > >>> >>>>>>>> may want > >> > > >>> >>>>>>>> >> to > >> > > >>> >>>>>>>> >> >> > edit > >> > > >>> >>>>>>>> >> >> > > > the > >> > > >>> >>>>>>>> >> >> > > > >>>> > config > >> > > >>> >>>>>>>> >> >> > > > >>>> > > as well to tell it the real ms IP. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike > >> > > Tutkowski" < > >> > > >>> >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > wrote: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > Here's what my > >> > /etc/network/interfaces > >> > > >>> file > >> > > >>> >>>>>>>> looks > >> > > >>> >>>>>>>> >> like, if > >> > > >>> >>>>>>>> >> >> > > that > >> > > >>> >>>>>>>> >> >> > > > >>>> is of > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > interest (the 192.168.233.0 > network > >> > is > >> > > the > >> > > >>> >>>>>>>> NAT network > >> > > >>> >>>>>>>> >> >> > VMware > >> > > >>> >>>>>>>> >> >> > > > >>>> Fusion > >> > > >>> >>>>>>>> >> >> > > > >>>> > set > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > up): > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto lo > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface lo inet loopback > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto eth0 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface eth0 inet manual > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto cloudbr0 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface cloudbr0 inet static > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > address 192.168.233.10 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > netmask 255.255.255.0 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > network 192.168.233.0 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > broadcast 192.168.233.255 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > dns-nameservers 8.8.8.8 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_ports eth0 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_fd 5 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_stp off > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_maxwait 1 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > post-up route add default gw > >> > > >>> >>>>>>>> 192.168.233.2 metric 1 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > pre-down route del default > gw > >> > > >>> >>>>>>>> 192.168.233.2 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08 > PM, > >> > Mike > >> > > >>> >>>>>>>> Tutkowski < > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > mike.tutkow...@solidfire.com> > >> wrote: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > You appear to be correct. > This is > >> > > from > >> > > >>> the > >> > > >>> >>>>>>>> MS log > >> > > >>> >>>>>>>> >> >> (below). > >> > > >>> >>>>>>>> >> >> > > > >>>> Discovery > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > timed > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > out. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I'm not sure why this would > be. > >> My > >> > > >>> network > >> > > >>> >>>>>>>> settings > >> > > >>> >>>>>>>> >> >> > > shouldn't > >> > > >>> >>>>>>>> >> >> > > > >>>> have > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > changed > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > since the last time I tried > this. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I am able to ping the KVM host > >> from > >> > > the > >> > > >>> MS > >> > > >>> >>>>>>>> host and > >> > > >>> >>>>>>>> >> vice > >> > > >>> >>>>>>>> >> >> > > > versa. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I'm even able to manually kick > >> off > >> > a > >> > > VM > >> > > >>> on > >> > > >>> >>>>>>>> the KVM > >> > > >>> >>>>>>>> >> host > >> > > >>> >>>>>>>> >> >> > and > >> > > >>> >>>>>>>> >> >> > > > >>>> ping from > >> > > >>> >>>>>>>> >> >> > > > >>>> > > it > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > to the MS host. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I am using NAT from my Mac OS > X > >> > host > >> > > >>> (also > >> > > >>> >>>>>>>> running > >> > > >>> >>>>>>>> >> the > >> > > >>> >>>>>>>> >> >> CS > >> > > >>> >>>>>>>> >> >> > > MS) > >> > > >>> >>>>>>>> >> >> > > > >>>> to the > >> > > >>> >>>>>>>> >> >> > > > >>>> > VM > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > running KVM (VMware Fusion). > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,141 DEBUG > >> > > >>> >>>>>>>> >> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer] > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > >> > > >>> :ctx-6b28dc48) > >> > > >>> >>>>>>>> Timeout, > >> > > >>> >>>>>>>> >> to > >> > > >>> >>>>>>>> >> >> > wait > >> > > >>> >>>>>>>> >> >> > > > for > >> > > >>> >>>>>>>> >> >> > > > >>>> the > >> > > >>> >>>>>>>> >> >> > > > >>>> > > host > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > connecting to mgt svr, > assuming > >> it > >> > is > >> > > >>> failed > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,144 WARN > >> > > >>> >>>>>>>> >> >> [c.c.r.ResourceManagerImpl] > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > >> > > >>> :ctx-6b28dc48) > >> > > >>> >>>>>>>> Unable to > >> > > >>> >>>>>>>> >> >> find > >> > > >>> >>>>>>>> >> >> > > the > >> > > >>> >>>>>>>> >> >> > > > >>>> server > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > resources at > >> http://192.168.233.10 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,145 INFO > >> > > >>> >>>>>>>> >> >> > [c.c.u.e.CSExceptionErrorCode] > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > >> > > >>> :ctx-6b28dc48) > >> > > >>> >>>>>>>> Could not > >> > > >>> >>>>>>>> >> >> find > >> > > >>> >>>>>>>> >> >> > > > >>>> exception: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > com.cloud.exception.DiscoveryException > >> > > >>> in > >> > > >>> >>>>>>>> error code > >> > > >>> >>>>>>>> >> >> list > >> > > >>> >>>>>>>> >> >> > > for > >> > > >>> >>>>>>>> >> >> > > > >>>> > > exceptions > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,147 WARN > >> > > >>> >>>>>>>> >> [o.a.c.a.c.a.h.AddHostCmd] > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > >> > > >>> :ctx-6b28dc48) > >> > > >>> >>>>>>>> Exception: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > com.cloud.exception.DiscoveryException: > >> > > >>> >>>>>>>> Unable to add > >> > > >>> >>>>>>>> >> >> the > >> > > >>> >>>>>>>> >> >> > > host > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > at > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > >> > > >>> >>>>>>>> >> > >> > > >>> >>>>>>>> > >> > > >>> > >> > > > >> > > >> > com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778) > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I do seem to be able to > telnet in > >> > > from > >> > > >>> my > >> > > >>> >>>>>>>> KVM host to > >> > > >>> >>>>>>>> >> >> the > >> > > >>> >>>>>>>> >> >> > MS > >> > > >>> >>>>>>>> >> >> > > > >>>> host's > >> > > >>> >>>>>>>> >> >> > > > >>>> > > 8250 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > port: > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet > >> > > >>> 192.168.233.1 > >> > > >>> >>>>>>>> 8250 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Trying 192.168.233.1... > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Connected to 192.168.233.1. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Escape character is '^]'. > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > -- > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > *Senior CloudStack Developer, > >> > SolidFire > >> > > >>> Inc.* > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > Advancing the way the world uses > >> the > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > cloud< > >> > > >>> >>>>>>>> >> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > *™* > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > -- > >> > > >>> >>>>>>>> >> >> > > > >>>> > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > >>>> > *Senior CloudStack Developer, > SolidFire > >> > > Inc.* > >> > > >>> >>>>>>>> >> >> > > > >>>> > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > >>>> > o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > >>>> > Advancing the way the world uses the > >> > > >>> >>>>>>>> >> >> > > > >>>> > cloud< > >> > > >>> >>>>>>>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> >> >> > > > >>>> > *™* > >> > > >>> >>>>>>>> >> >> > > > >>>> > > >> > > >>> >>>>>>>> >> >> > > > >>>> > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >>> -- > >> > > >>> >>>>>>>> >> >> > > > >>> *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > >>> *Senior CloudStack Developer, SolidFire > >> > Inc.* > >> > > >>> >>>>>>>> >> >> > > > >>> e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > >>> o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > >>> Advancing the way the world uses the > >> cloud< > >> > > >>> >>>>>>>> >> >> > > > > >> > > http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> >> >> > > > >>> *™* > >> > > >>> >>>>>>>> >> >> > > > >>> > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > >> -- > >> > > >>> >>>>>>>> >> >> > > > >> *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > >> *Senior CloudStack Developer, SolidFire > >> Inc.* > >> > > >>> >>>>>>>> >> >> > > > >> e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > >> o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > >> Advancing the way the world uses the > cloud< > >> > > >>> >>>>>>>> >> >> > > > > >> > > http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> >> >> > > > >> *™* > >> > > >>> >>>>>>>> >> >> > > > >> > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > -- > >> > > >>> >>>>>>>> >> >> > > > > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > > *Senior CloudStack Developer, SolidFire > >> Inc.* > >> > > >>> >>>>>>>> >> >> > > > > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > > o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > > Advancing the way the world uses the > cloud< > >> > > >>> >>>>>>>> >> >> > > > > >> > > http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> >> >> > > > > *™* > >> > > >>> >>>>>>>> >> >> > > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > -- > >> > > >>> >>>>>>>> >> >> > > > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > > > *Senior CloudStack Developer, SolidFire > Inc.* > >> > > >>> >>>>>>>> >> >> > > > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > > > o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > > > Advancing the way the world uses the > >> > > >>> >>>>>>>> >> >> > > > cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> >> >> > > > *™* > >> > > >>> >>>>>>>> >> >> > > > > >> > > >>> >>>>>>>> >> >> > > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > -- > >> > > >>> >>>>>>>> >> >> > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>>>>> >> >> > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> >> > o: 303.746.7302 > >> > > >>> >>>>>>>> >> >> > Advancing the way the world uses the > >> > > >>> >>>>>>>> >> >> > cloud< > >> > > http://solidfire.com/solution/overview/?video=play > >> > > >>> > > >> > > >>> >>>>>>>> >> >> > *™* > >> > > >>> >>>>>>>> >> >> > > >> > > >>> >>>>>>>> >> >> > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > > >> > > >>> >>>>>>>> >> > -- > >> > > >>> >>>>>>>> >> > *Mike Tutkowski* > >> > > >>> >>>>>>>> >> > *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>>>>> >> > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> >> > o: 303.746.7302 > >> > > >>> >>>>>>>> >> > Advancing the way the world uses the > >> > > >>> >>>>>>>> >> > cloud< > >> > http://solidfire.com/solution/overview/?video=play > >> > > > > >> > > >>> >>>>>>>> >> > *™* > >> > > >>> >>>>>>>> >> > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > > >> > > >>> >>>>>>>> > -- > >> > > >>> >>>>>>>> > *Mike Tutkowski* > >> > > >>> >>>>>>>> > *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>>>>> > e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>>> > o: 303.746.7302 > >> > > >>> >>>>>>>> > Advancing the way the world uses the > >> > > >>> >>>>>>>> > cloud< > >> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>>> > *™* > >> > > >>> >>>>>>>> > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> > >> > > >>> >>>>>>> -- > >> > > >>> >>>>>>> *Mike Tutkowski* > >> > > >>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>>>> e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>>> o: 303.746.7302 > >> > > >>> >>>>>>> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>>> *™* > >> > > >>> >>>>>>> > >> > > >>> >>>>>> > >> > > >>> >>>>>> > >> > > >>> >>>>>> > >> > > >>> >>>>>> -- > >> > > >>> >>>>>> *Mike Tutkowski* > >> > > >>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>>> e: mike.tutkow...@solidfire.com > >> > > >>> >>>>>> o: 303.746.7302 > >> > > >>> >>>>>> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>>> *™* > >> > > >>> >>>>>> > >> > > >>> >>>>> > >> > > >>> >>>>> > >> > > >>> >>>>> > >> > > >>> >>>>> -- > >> > > >>> >>>>> *Mike Tutkowski* > >> > > >>> >>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>>> e: mike.tutkow...@solidfire.com > >> > > >>> >>>>> o: 303.746.7302 > >> > > >>> >>>>> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>>> *™* > >> > > >>> >>>>> > >> > > >>> >>>> > >> > > >>> >>>> > >> > > >>> >>>> > >> > > >>> >>>> -- > >> > > >>> >>>> *Mike Tutkowski* > >> > > >>> >>>> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>>> e: mike.tutkow...@solidfire.com > >> > > >>> >>>> o: 303.746.7302 > >> > > >>> >>>> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>>> *™* > >> > > >>> >>>> > >> > > >>> >>> > >> > > >>> >>> > >> > > >>> >>> > >> > > >>> >>> -- > >> > > >>> >>> *Mike Tutkowski* > >> > > >>> >>> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >>> e: mike.tutkow...@solidfire.com > >> > > >>> >>> o: 303.746.7302 > >> > > >>> >>> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >>> *™* > >> > > >>> >>> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> -- > >> > > >>> >> *Mike Tutkowski* > >> > > >>> >> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> >> e: mike.tutkow...@solidfire.com > >> > > >>> >> o: 303.746.7302 > >> > > >>> >> Advancing the way the world uses the cloud< > >> > > >>> http://solidfire.com/solution/overview/?video=play> > >> > > >>> >> *™* > >> > > >>> >> > >> > > >>> > > >> > > >>> > > >> > > >>> > > >> > > >>> > -- > >> > > >>> > *Mike Tutkowski* > >> > > >>> > *Senior CloudStack Developer, SolidFire Inc.* > >> > > >>> > e: mike.tutkow...@solidfire.com > >> > > >>> > o: 303.746.7302 > >> > > >>> > Advancing the way the world uses the > >> > > >>> > cloud<http://solidfire.com/solution/overview/?video=play> > >> > > >>> > *™* > >> > > >>> > >> > > >> > >> > > >> > >> > > >> > >> > > >> -- > >> > > >> *Mike Tutkowski* > >> > > >> *Senior CloudStack Developer, SolidFire Inc.* > >> > > >> e: mike.tutkow...@solidfire.com > >> > > >> o: 303.746.7302 > >> > > >> Advancing the way the world uses the cloud< > >> > > http://solidfire.com/solution/overview/?video=play> > >> > > >> *™* > >> > > >> > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > *Mike Tutkowski* > >> > > > *Senior CloudStack Developer, SolidFire Inc.* > >> > > > e: mike.tutkow...@solidfire.com > >> > > > o: 303.746.7302 > >> > > > Advancing the way the world uses the > >> > > > cloud<http://solidfire.com/solution/overview/?video=play> > >> > > > *™* > >> > > > >> > > >> > > >> > > >> > -- > >> > *Mike Tutkowski* > >> > *Senior CloudStack Developer, SolidFire Inc.* > >> > e: mike.tutkow...@solidfire.com > >> > o: 303.746.7302 > >> > Advancing the way the world uses the > >> > cloud<http://solidfire.com/solution/overview/?video=play> > >> > *™* > >> > > >> > > > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud<http://solidfire.com/solution/overview/?video=play> > > *™* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *™*