ikoula opened a new issue #2695: Unable to create local link issue (CS 4.11.1-rc1 + XCP-ng 7.4.1) URL: https://github.com/apache/cloudstack/issues/2695 <!-- Verify first that your issue/request is not already reported on GitHub. Also test if the latest release and master branch are affected too. Always add information AFTER of these HTML comments, but no need to delete the comments. --> ##### ISSUE TYPE <!-- Pick one below and delete the rest --> * Bug Report ##### COMPONENT NAME <!-- Categorize the issue, e.g. API, VR, VPN, UI, etc. --> ~~~ Hypervisor controler ~~~ ##### CLOUDSTACK VERSION <!-- New line separated list of affected versions, commit ID for issues on master branch. --> ~~~ CS Version : 4.11.1-rc1 (fresh install) ~~~ ##### CONFIGURATION <!-- Information about the configuration if relevant, e.g. basic network, advanced networking, etc. N/A otherwise --> Management Server : CentOS 7 (fresh install) Hypervisor : 2 x XCP-ng 7.4.1 (fresh install, in the same pool) Zone : Advanced without SG (local storage enabled for both system & user VMs) ##### SUMMARY <!-- Explain the problem/feature briefly --> Unable to create the local link network on the newly added host AddHost API Call is succeeding but host stays in alert state since the local link cannot be created. ##### STEPS TO REPRODUCE <!-- For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate. For new features, show how the feature would be used. --> <!-- Paste example playbooks or commands between quotes below --> ~~~ Add an XCP-ng 7.4.1 host under an Advanced Zone (I think the issue will also hit basic zones). ~~~ <!-- You can also paste gist.github.com links for larger files --> ##### EXPECTED RESULTS <!-- What did you expect to happen when running the steps above? --> ~~~ local link creation should be successful. ~~~ ##### ACTUAL RESULTS <!-- What actually happened? --> ~~~ 2018-06-05 11:19:44,110 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-2:ctx-ed8cfa5d) (logid:8c5c169f) Unable to create local link network The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem. at com.xensource.xenapi.Types.checkResponse(Types.java:1609) at com.xensource.xenapi.Connection.dispatch(Connection.java:395) at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457) at com.xensource.xenapi.VIF.plug(VIF.java:887) at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.plugDom0Vif(CitrixResourceBase.java:4056) at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4806) at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:52) at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:38) at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1728) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ~~~ Here is a more complete snippet from the management log during the addHost : [addhost_xcp-ng_741.log](https://github.com/apache/cloudstack/files/2075157/addhost_xcp-ng_741.log) <!-- Paste verbatim command output between quotes below --> xenopds seems to be the culprit : ~~~ [root@test-xen01 ~]# xe vif-param-list uuid=95e50cbb-80ce-692a-ef33-a3055b4e8b00 uuid ( RO) : 95e50cbb-80ce-692a-ef33-a3055b4e8b00 vm-uuid ( RO): 4313460b-55fe-40b0-afc3-7b1a3ab63cfd vm-name-label ( RO): Control domain on host: test-xen01 allowed-operations (SRO): attach; plug current-operations (SRO): device ( RO): 0 MAC ( RO): FE:FF:FF:FF:FF:FF MAC-autogenerated ( RO): false MTU ( RO): 0 currently-attached ( RO): false qos_algorithm_type ( RW): qos_algorithm_params (MRW): qos_supported_algorithms (SRO): other-config (MRW): nameLabel: link_local_network_vif network-uuid ( RO): b7c29721-af1b-d09d-aeb5-a6e43e8062a6 network-name-label ( RO): cloud_link_local_network io_read_kbs ( RO): <unknown> io_write_kbs ( RO): <unknown> locking-mode ( RW): network_default ipv4-allowed (SRW): ipv6-allowed (SRW): ipv4-configuration-mode ( RO): None ipv4-addresses ( RO): ipv4-gateway ( RO): ipv6-configuration-mode ( RO): None ipv6-addresses ( RO): ipv6-gateway ( RO): [root@test-xen01 ~]# xe vif-plug uuid=95e50cbb-80ce-692a-ef33-a3055b4e8b00 The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem. message: Object with type VM and id 4313460b-55fe-40b0-afc3-7b1a3ab63cfd does not exist in xenopsd [root@cs-preprod-z1-p1-c1-xen01 ~]# systemctl status xenopsd-xc.service ● xenopsd-xc.service - VM manager for Xen using libxc Loaded: loaded (/usr/lib/systemd/system/xenopsd-xc.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/xenopsd-xc.service.d └─local.conf, slice.conf Active: active (running) since Mon 2018-06-04 17:41:19 CEST; 5min ago Docs: man:xenopsd-xc(1) Main PID: 28505 (xenopsd-xc) CGroup: /control.slice/xenopsd-xc.service └─28505 /usr/sbin/xenopsd-xc --config /etc/xenopsd.conf [root@cs-preprod-z1-p1-c1-xen01 ~]# systemctl status xenopsd-xenlight.service ● xenopsd-xenlight.service - VM manager for Xen using libxenlight Loaded: loaded (/usr/lib/systemd/system/xenopsd-xenlight.service; disabled; vendor preset: disabled) Active: active (running) since Mon 2018-06-04 15:49:52 CEST; 1h 56min ago Docs: man:xenopsd-xenlight(1) Main PID: 8317 (xenopsd-xenligh) CGroup: /system.slice/xenopsd-xenlight.service └─8317 /usr/sbin/xenopsd-xenlight --config /etc/xenopsd.conf ~~~ We are able to experiment in our lab environment as needed (including CS builds/reinstall or XCP reinstalls).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
