Some times ago, the cluster worked. Here are the entries in the DB :
mysql> select id,name,uuid,guid,pod_id,data_center_id,hypervisor_type,allocation_state,engine_state from cloud.cluster where id=16; +----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+ | id | name | uuid | guid | pod_id | data_center_id | hypervisor_type | allocation_state | engine_state | +----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+ | 16 | vcenter1/datacenter1/cluster1 | a95659a5-8918-438d-a3c9-214e8fcdcd97 | c74d97b0-1eae-357e-84aa-9d5bade97baf | 1 | 1 | VMware | Enabled | Disabled | +----+--------------------------------+--------------------------------------+--------------------------------------+--------+----------------+-----------------+------------------+--------------+ mysql> select * from cloud.cluster_details where cluster_id="16"; +------+------------+----------------------------+---------------------------------------------------------+ | id | cluster_id | name | value | +------+------------+----------------------------+---------------------------------------------------------+ | 1048 | 16 | memoryOvercommitRatio | 1.0 | | 1049 | 16 | guestvswitchtype | vmwaresvs | | 1050 | 16 | username | csadmin@vsphere.local | | 1051 | 16 | cpuOvercommitRatio | 1.0 | | 1052 | 16 | password | ANON | | 1053 | 16 | guestvswitchname | vSwitch0 | | 1055 | 16 | url | http://vcenter1/datacenter1/cluster1 | | 1063 | 16 | NativeHA | false | | 1065 | 16 | outOfBandManagementEnabled | false | | 1066 | 16 | resourceHAEnabled | true | +------+------------+----------------------------+---------------------------------------------------------+ mysql> select * from cloud.vmware_data_center where id=4; +----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+ | id | uuid | name | guid | vcenter_host | username | password | +----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+ | 4 | c39fe7c5-3478-43fa-aed1-38d40072279d | datacenter1 | datacenter1@vcenter | vcenter1 | csadmin@vsphere.local | ANON | +----+--------------------------------------+-------------------+----------------------------------+----------------+-----------------------+----------------------------------+ mysql> SELECT id,name,status,guid FROM cloud.host WHERE hypervisor_type = 'VMware' AND status != 'Removed'; +----+-------+--------+------------------------------------+ | id | name | status | guid | +----+-------+--------+------------------------------------+ | 65 | esxi1 | Up | HostSystem:host-973@vcenter1 | +----+-------+--------+------------------------------------+ Host 65 is the ESXi hypervisor. Is the log bellow from a version previous to 4.11 ? Florent ----- Mail original ----- > De: "Sergey Levitskiy" <serg...@hotmail.com> > À: "users" <users@cloudstack.apache.org> > Envoyé: Mardi 16 Avril 2019 22:59:46 > Objet: Re: Problem with ACS 4.11.2 and vCenter server > It doesn’t look you have properly added VMware cluster & respective storage. > Can > you try again and post management server log which should have entries similar > to below after you added cluster in UI. First you need to add VMware DC and > then cluster. > BTW you also have issues with one of your KVM hosts id=65. > > > 2017-10-31 12:17:37,116 DEBUG [c.c.a.ApiServlet] > (catalina-exec-22:ctx-f2dc50b2) (logid:799ac7c1) ===START=== > 172.17.101.246 -- POST command=addVmwareDc&response=json > 2017-10-31 12:17:37,132 INFO [c.c.h.v.m.VmwareManagerImpl] > (catalina-exec-22:ctx-f2dc50b2 ctx-a324e697) (logid:799ac7c1) Ignoring API > call addVmwareDc, because VMware DC 172.17.101.246/WeDatacenter is already > associated with specified zone with id 1 > 2017-10-31 12:17:37,133 DEBUG [c.c.a.ApiServlet] > (catalina-exec-22:ctx-f2dc50b2 ctx-a324e697) (logid:799ac7c1) ===END=== > 172.17.101.246 -- POST command=addVmwareDc&response=json > 2017-10-31 12:17:37,155 DEBUG [c.c.a.ApiServlet] > (catalina-exec-20:ctx-a26bd8ef) (logid:272077ae) ===START=== > 172.17.101.246 -- GET > command=addCluster&zoneId=f538e730-7431-4c65-bf7d-eccc154e8b48&hypervisor=VMware&clustertype=ExternalManaged&podId=6262dca1-59df-4a32-b80a-52435f2ec4f7&username=administrator%40vsphere.local&url=http%3A%2F%2F172.17.101.246%2FWeDatacenter%2FWeCluster&clustername=172.17.101.246%2FWeDatacenter%2FWeCluster&response=json&_=1509432458636 > 2017-10-31 12:17:37,225 INFO [c.c.h.v.VmwareServerDiscoverer] > (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Discover > host. dc: 1, pod: 1, cluster: 7, uri host: 172.17.101.246 > 2017-10-31 12:17:37,313 INFO [c.c.h.v.VmwareServerDiscoverer] > (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Detected > private network label : DSwitch,,vmwaresvs > 2017-10-31 12:17:37,313 DEBUG [c.c.h.v.r.VmwareContextFactory] > (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) initialize > VmwareContext. url: https://172.17.101.246/sdk/vimService, username: > administrator@vsphere.local, password: W********* > 2017-10-31 12:17:37,535 INFO [c.c.h.v.u.VmwareContext] > (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) New > VmwareContext object, current outstanding count: 1 > 2017-10-31 12:17:37,662 INFO [c.c.h.v.m.VmwareManagerImpl] > (catalina-exec-20:ctx-a26bd8ef ctx-06a2c997) (logid:272077ae) Preparing > network on host com.cloud.hypervisor.vmware.util.VmwareContext@53bf4bc2 for > DSwitch,,vmwaresvs > > > > > > >On 4/16/19, 11:48 AM, "Florent Paillot" <florent.pail...@inria.fr> wrote: > > This is the full log : > > https://filesender.renater.fr/?s=download&token=6be77301-3893-46fa-912c-3912086cae5f > > Florent > > ----- Mail original ----- > > De: "Sergey Levitskiy" <serg...@hotmail.com> > > À: "users" <users@cloudstack.apache.org> > > Envoyé: Mardi 16 Avril 2019 17:27:37 > > Objet: Re: Problem with ACS 4.11.2 and vCenter server > > > Please post a full management-server.log without parsing. > > > >On 4/16/19, 7:49 AM, "Florent Paillot" <florent.pail...@inria.fr> wrote: > > > > I did this on purpose to limit the size of the log. > > > > I let the controller run for more than 30mins. Can't find any > revelant error... > > :( > > I put some grep results here : > https://pad.inria.fr/p/np_IvcpeIGM4BncZEM1 > > > > On the production controller (4.9.3), if we have connectivity problem > with the > > vCenter, I see log with the "com.cloud.hypervisor.vmware.*" context. > > Is there something to change in the log4j config to see this logs in > 4.11 ? > > > > The vim25.jar is the right one, after unziping > > "/usr/share/cloudstack-management/lib/cloudstack-4.11.2.0.jar" in > > com/vmware/vim25/vim.wsdl, I see " <soap:operation > soapAction="urn:vim25/6.7" > > style="document" />". > > > > > > Florent > > > > ----- Mail original ----- > > > De: "Suresh Kumar Anaparti" <sureshkumar.anapa...@gmail.com> > > > À: "users" <users@cloudstack.apache.org> > > > Envoyé: Mardi 16 Avril 2019 14:22:28 > > > Objet: Re: Problem with ACS 4.11.2 and vCenter server > > > > > Florent, The log doesn't have any CS to vCenter connectivity > details. The > > > CS management server was started and stopped in 2 mins. It needs > some time > > > to load modules and connect to the hosts and storages. Wait for > some more > > > time for the connectivity details in the log. > > > > > > You can install vim25.jar for 6.7 version and give a try. > > > > > > - Suresh > > > > > > On Tue, Apr 16, 2019 at 5:34 PM Florent Paillot > <florent.pail...@inria.fr> > > > wrote: > > > > > >> Here is the log : https://pad.inria.fr/p/np_IvcpeIGM4BncZEM1 > > >> It's a complete 3 minutes start/run/stop. > > >> > > >> Thanks for the info about "cs.vmware.api.version*". For 4.11.2, it > seems > > >> that vSphere 6.7 is the target : > > >> pom.xml: <cs.vmware.api.version>6.7</cs.vmware.api.version> > > >> > > >> > > >> Florent > > >> > > >> ----- Mail original ----- > > >> > De: "Suresh Kumar Anaparti" <sureshkumar.anapa...@gmail.com> > > >> > À: "users" <users@cloudstack.apache.org> > > >> > Envoyé: Mardi 16 Avril 2019 13:05:20 > > >> > Objet: Re: Problem with ACS 4.11.2 and vCenter server > > >> > > >> > Hi Florent, > > >> > > > >> > Log is missing. I think attachments are ignored. Either provide > a link to > > >> > access the log or paste the logs in email (few logs of the > issue). > > >> > > > >> > Irrespective of the vCenter version (hard coded in the build > param > > >> > *cs.vmware.api.version*), CS would try to connect to the vCenter > server. > > >> > > > >> > - Suresh > > >> > > > >> > > > >> > On Tue, Apr 16, 2019 at 2:44 PM Florent Paillot < > > >> florent.pail...@inria.fr> > > >> > wrote: > > >> > > > >> >> Thanks for the answers, > > >> >> > > >> >> Paul : I know that the vCenter 6.7 is not officialy supported > but my > > >> >> colleague reported that it worked with 4.9.3 . > > >> >> When you say "cloudstack won't recognise it", is it hard coded > ? If the > > >> >> the vCenter auth API doesn't change between 6.5 and 6.7 it > should work > > >> or > > >> >> at least try to connect to the vCenter. > > >> >> > > >> >> I'm agree with you Suresh, even if CS can't communicate with > the vCenter > > >> >> we should see something in the logs or in the packet capture. > > >> >> > > >> >> I am able to ping and telnet from the CS controller to the > vCenter on > > >> port > > >> >> 443 (and I see the packet with tcpdump). > > >> >> > > >> >> The log is in attachement. > > >> >> > > >> >> Florent > > >> >> > > >> >> > > >> >> ----- Mail original ----- > > >> >> > De: "Suresh Kumar Anaparti" <sureshkumar.anapa...@gmail.com> > > >> >> > À: "users" <users@cloudstack.apache.org> > > >> >> > Envoyé: Lundi 15 Avril 2019 17:54:00 > > >> >> > Objet: Re: Problem with ACS 4.11.2 and vCenter server > > >> >> > > >> >> > Hi Florent, What was the CS behavior & vCenter/ESXi hosts > connectivity > > >> >> > status before upgrade to 4.11.2 from 4.9.3? Have you upgraded > vCenter > > >> as > > >> >> > well? Are you able to ping vCenter from CS management server? > > >> >> > > > >> >> > Even vCenter 6.7 is not recognised, CS would try to connect > with the > > >> >> > details specified, which is logged. Have you enabled debug > logs? > > >> >> > > > >> >> > - Suresh > > >> >> > > > >> >> > On Mon, Apr 15, 2019 at 7:38 PM Paul Angus > <paul.an...@shapeblue.com> > > >> >> wrote: > > >> >> > > > >> >> >> Ah. vCenter 6.7 isn't supported yet, cloudstack won't > recognise it. > > >> >> >> > > >> >> >> paul.an...@shapeblue.com > > >> >> >> www.shapeblue.com > > >> >> >> Amadeus House, Floral Street, London WC2E 9DPUK > > >> >> >> @shapeblue > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> -----Original Message----- > > >> >> >> From: Florent Paillot <florent.pail...@inria.fr> > > >> >> >> Sent: 15 April 2019 14:26 > > >> >> >> To: users <users@cloudstack.apache.org> > > >> >> >> Subject: Re: Problem with ACS 4.11.2 and vCenter server > > >> >> >> > > >> >> >> Hi Suresh, > > >> >> >> The vCenter is a 6.7u2 (last build). > > >> >> >> > > >> >> >> There is no issue log, that the problem... I don't see any > error > > >> message > > >> >> >> concerning a failed connection to the vCenter and CS does > not seems > > >> to > > >> >> send > > >> >> >> any packet to the vCenter. > > >> >> >> > > >> >> >> Florent > > >> >> >> > > >> >> >> > > >> >> >> ----- Mail original ----- > > >> >> >> > De: "Suresh Kumar Anaparti" > <sureshkumar.anapa...@gmail.com> > > >> >> >> > À: "users" <users@cloudstack.apache.org> > > >> >> >> > Envoyé: Lundi 15 Avril 2019 15:10:44 > > >> >> >> > Objet: Re: Problem with ACS 4.11.2 and vCenter server > > >> >> >> > > >> >> >> > Hi Florent, > > >> >> >> > > > >> >> >> > Can you share the vCenter version and build details. Also, > the > > >> >> >> > cloudstack management server log with the issue logs while > > >> connecting > > >> >> to > > >> >> >> vCenter. > > >> >> >> > > > >> >> >> > - Suresh > > >> >> >> > > > >> >> >> > On Mon, Apr 15, 2019 at 6:24 PM Paul Angus < > > >> paul.an...@shapeblue.com> > > >> >> >> wrote: > > >> >> >> > > > >> >> >> >> Assuming that you have updated the core code to recognise > and use > > >> >> >> 'newer' > > >> >> >> >> libraries/SDKs than the ones currently in use (and > thoroughly > > >> tested > > >> >> >> >> it) then, please do create a pull request. Otherwise, I > > >> personally > > >> >> >> >> would advise that you leave things 'as-is' to get the > most stable > > >> >> >> experience. > > >> >> >> >> > > >> >> >> >> I don't *think* that you need to run > > >> >> >> >> > > >> >> >> >> mvn clean > > >> >> >> >> mvn install -Dnoredist > > >> >> >> >> > > >> >> >> >> *After* installing the non-oss components. > > >> >> >> >> > > >> >> >> >> You might want to do a mvn clean before you start. > > >> >> >> >> > > >> >> >> >> It's been a while since I've done it manually now. > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> paul.an...@shapeblue.com > > >> >> >> >> www.shapeblue.com > > >> >> >> >> Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> -----Original Message----- > > >> >> >> >> From: Florent Paillot <florent.pail...@inria.fr> > > >> >> >> >> Sent: 15 April 2019 13:17 > > >> >> >> >> To: users <users@cloudstack.apache.org> > > >> >> >> >> Subject: Re: Problem with ACS 4.11.2 and vCenter server > > >> >> >> >> > > >> >> >> >> Yes, this is the list of the files in deps/ : > > >> >> >> >> cloud-iControl.jar cloud-netscaler-sdx.jar > iControl-12.1.0.jar > > >> >> >> >> install-non-oss.sh manageontap.jar vim25_51.jar > vim25_55.jar > > >> >> >> >> vim25_60.jar vim25_65.jar vim25_67.jar > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> I had to add the following lines to "install-non-oss.sh" > to get > > >> the > > >> >> >> build > > >> >> >> >> pass : > > >> >> >> >> > > >> >> >> >> mvn install:install-file -Dfile=vim25_67.jar > > >> >> >> >> -DgroupId=com.cloud.com.vmware -DartifactId=vmware-vim25 > > >> >> >> -Dversion=6.7 > > >> >> >> >> -Dpackaging=jar > > >> >> >> >> mvn install:install-file -Dfile=iControl-12.1.0.jar > > >> >> >> >> -DgroupId=com.cloud.com.f5 -DartifactId=icontrol > > >> >> >> >> -Dversion=12.1 -Dpackaging=jar > > >> >> >> >> > > >> >> >> >> I can open a PR or an issue if it makes sense. > > >> >> >> >> > > >> >> >> >> Florent > > >> >> >> >> > > >> >> >> >> -- > > >> >> >> >> Florent PAILLOT > > >> >> >> >> SESI - Cellule architecture > > >> >> >> >> > > >> >> >> >> INRIA Bordeaux -Sud-Ouest > > >> >> >> >> 05.24.57.41.25 > > >> >> >> >> > > >> >> >> >> ----- Mail original ----- > > >> >> >> >> > De: "Paul Angus" <paul.an...@shapeblue.com> > > >> >> >> >> > À: "users" <users@cloudstack.apache.org> > > >> >> >> >> > Envoyé: Lundi 15 Avril 2019 14:10:21 > > >> >> >> >> > Objet: RE: Problem with ACS 4.11.2 and vCenter server > > >> >> >> >> > > >> >> >> >> > When packaging, did you install all of the nonoss > libraries > > >> into > > >> >> the > > >> >> >> >> > build before packaging? including vmware-vim25-6.5.jar > > >> >> >> >> > > > >> >> >> >> > paul.an...@shapeblue.com > > >> >> >> >> > www.shapeblue.com > > >> >> >> >> > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > -----Original Message----- > > >> >> >> >> > From: Dag Sonstebo <dag.sonst...@shapeblue.com> > > >> >> >> >> > Sent: 15 April 2019 12:43 > > >> >> >> >> > To: users@cloudstack.apache.org > > >> >> >> >> > Subject: Re: Problem with ACS 4.11.2 and vCenter server > > >> >> >> >> > > > >> >> >> >> > Florent, > > >> >> >> >> > > > >> >> >> >> > Not aware of any issues but I'm no packaging expert so > will let > > >> >> others > > >> >> >> >> > respond to this. > > >> >> >> >> > > > >> >> >> >> > However - any reason you're not just using the RPM > repositories > > >> >> rather > > >> >> >> >> > than packaging yourself? > > >> >> >> >> > > > >> >> >> >> > Regards, > > >> >> >> >> > Dag Sonstebo > > >> >> >> >> > Cloud Architect > > >> >> >> >> > ShapeBlue > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> >On 15/04/2019, 11:46, "Florent Paillot" < > > >> florent.pail...@inria.fr> > > >> >> >> >> wrote: > > >> >> >> >> > > > >> >> >> >> > Hi everyone, > > >> >> >> >> > We have a problem with our test environment running > ACS > > >> 4.11.2 > > >> >> on > > >> >> >> >> CentOS 7, not > > >> >> >> >> > communicating with a vCenter (upgraded from 4.9.3 to > 4.11.2 > > >> and > > >> >> >> >> CentOS 6 o > > >> >> >> >> > CentOS 7). We have no problems with our KVM host. > > >> >> >> >> > Our production is using 4.9.3 o, Centos 6 and > everything is > > >> OK. > > >> >> >> >> > > > >> >> >> >> > I used tcpdump and see nothing going out from the > management > > >> >> server > > >> >> >> >> to the > > >> >> >> >> > vCenter or the ESXi host. > > >> >> >> >> > > > >> >> >> >> > Could it be a package building problem or a vmware > module > > >> >> loading? > > >> >> >> >> > > > >> >> >> >> > I used the following commands to build the packages : > > >> >> >> >> > install-non-oss.sh > > >> >> >> >> > mvn clean > > >> >> >> >> > mvn install -Dnoredist > > >> >> >> >> > package.sh -p noredist -d centos7 > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > On the production server I see a : > > >> >> >> >> > > > >> >> >> >> > > >> >> >> > > >> >> > > >> > /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/vmware-vim25-6.0.jar > > >> >> >> >> > file. > > >> >> >> >> > I can't find it on the test environnement, is it > linked to > > >> >> jetty ? > > >> >> >> >> > > > >> >> >> >> > Any help is welcome :) > > >> >> >> >> > Regards, > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > dag.sonst...@shapeblue.com > > >> >> >> >> > www.shapeblue.com > > >> >> >> >> > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue