Milan Zamazal <mzama...@redhat.com> writes: > Vojtech Juranek <vjura...@redhat.com> writes: > >> Hi, >> OST is constantly failing today in test_verify_add_hosts with >> >> 19:34:33 E assert find_up_host() -> False != True after 1800 seconds >> 19:34:33 E + where find_up_host() -> False != True after 1800 >> seconds = <function true_within at 0x7fa62ac860d0>(<function >> test_verify_add_hosts.<locals>.find_up_host at >> 0x7fa628f97940>, timeout=1800) >> 19:34:33 E + where <function true_within at 0x7fa62ac860d0> = >> assert_utils.true_within >> 19:34:33 E + and 1800 = constants.ADD_HOST_TIMEOUT >> >> See e.g. https://redir.apps.ovirt.org/dj/job/ds-ost-baremetal_manual/23661 >> >> Any idea what's wrong? > > No. From another failed run: > > The host installation succeeded: > > 2021-12-16 21:23:45,073+01 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (EE-ManagedThreadFactory-engine-Thread-1) > [5b20f413-4877-49c3-873a-ef0846b14cca] EVENT_ID: > ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Installing Host > ost-basic-suite-master-host-0. restart libvirtd. > ... > 2021-12-16 21:23:48,105+01 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (EE-ManagedThreadFactory-engine-Thread-1) > [5b20f413-4877-49c3-873a-ef0846b14cca] EVENT_ID: > VDS_ANSIBLE_INSTALL_FINISHED(561), Ansible host-deploy playbook > execution has successfully finished on host > ost-basic-suite-master-host-0. > > And then start repeating tracebacks in engine.log, e.g.: > > 2021-12-16 21:52:48,301+01 INFO > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp > Reactor) [5b20f413-4877-49c3-873a-ef0846b14cca] Connecting to > ost-basic-suite-master-host-0/192.168.206.3 > 2021-12-16 21:52:48,301+01 INFO > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp > Reactor) [5b20f413-4877-49c3-873a-ef0846b14cca] Connected to > ost-basic-suite-master-host-0/192.168.206.3:54321 > 2021-12-16 21:52:48,301+01 ERROR > [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] > (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-80) > [] Unable to RefreshCapabilities: ConnectException: Connection refused > 2021-12-16 21:52:48,301+01 DEBUG
I also get IPv6 network errors when running locally. When I run it with IPv4 only then it works. > [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] > (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-80) > [] Exception: > org.ovirt.engine.core.vdsbroker.TransportRunTimeException: Connection > issues during send request > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:317) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand.executeVdsBrokerCommand(GetCapabilitiesAsyncVDSCommand.java:30) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65) > at > org.ovirt.engine.core.dal//org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:409) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown > Source) > at jdk.internal.reflect.GeneratedMethodAccessor238.invoke(Unknown > Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78) > at > org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) > at jdk.internal.reflect.GeneratedMethodAccessor55.invoke(Unknown Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79) > at > org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown > Source) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:782) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refreshCapabilities(HostMonitoring.java:187) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:146) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refresh(HostMonitoring.java:103) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VdsManager.refreshImpl(VdsManager.java:302) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VdsManager.refresh(VdsManager.java:273) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at > java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) > at > org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360) > at > org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > at > org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227) > Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: > Connection failed > at > org.ovirt.vdsm-jsonrpc-java//org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:146) > at > org.ovirt.vdsm-jsonrpc-java//org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:137) > at > org.ovirt.vdsm-jsonrpc-java//org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:105) > at > deployment.engine.ear//org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:315) > ... 38 more > Caused by: java.net.ConnectException: Connection refused > at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) > at > org.ovirt.vdsm-jsonrpc-java//org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:120) > ... 41 more > > On the host side, Vdsm is restarted > > 2021-12-16 21:23:43,151+0100 INFO (MainThread) [vds] Received signal 15, > shutting down (vdsmd:66) > 2021-12-16 21:23:54,048+0100 INFO (MainThread) [vds] (PID: 34647) I am > the actual vdsm 4.50.0.3.32.git467747f7a ost-basic-suite-master-host-0 > (4.18.0-348.2.1.el8_5.x86_64) (vdsmd:153) > > and then it looks like normally running. _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DPI7I67SFEKVPRC5CLXQRAFBI3C55BAS/
