> On 21 Dec 2016, at 16:25, Yaniv Kaul <[email protected]> wrote:
>
>
>
> On Wed, Dec 21, 2016 at 5:19 PM, Michal Skrivanek
> <[email protected] <mailto:[email protected]>> wrote:
>
>> On 21 Dec 2016, at 14:56, Michal Skrivanek <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>
>>> On 21 Dec 2016, at 12:19, Eyal Edri <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>
>>>
>>> On Wed, Dec 21, 2016 at 12:56 PM, Vinzenz Feenstra <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>> On Dec 21, 2016, at 11:17 AM, Barak Korren <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>> The test for running VMs had been failing since yesterday.
>>>>
>>>> The patch merged before the failures started was:
>>>> https://gerrit.ovirt.org/#/c/68826/ <https://gerrit.ovirt.org/#/c/68826/>
>>>
>>>
>>>
>>>>
>>>> The error we`re seeing is a time-out (after two minutes) while running
>>>> this API call:
>>>>
>>>> api.vms.get(VM0_NAME).status.state == ‘up'
>>>
>>> This is a REST API call, the patch above is Frontend. So this is unrelated.
>>>
>>> However on Host 0 I can see this:
>>>
>>> 2016-12-20 16:54:43,544 ERROR (vm/d299ab29) [virt.vm]
>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') The vm start process failed
>>> (vm:615)
>>> Traceback (most recent call last):
>>> File "/usr/share/vdsm/virt/vm.py", line 551, in _startUnderlyingVm
>>> self._run()
>>> File "/usr/share/vdsm/virt/vm.py", line 1991, in _run
>>> self._connection.createXML(domxml, flags),
>>> File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
>>> 123, in wrapper
>>> ret = f(*args, **kwargs)
>>> File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 941, in
>>> wrapper
>>> return func(inst, *args, **kwargs)
>>> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3782, in
>>> createXML
>>> if ret is None:raise libvirtError('virDomainCreateXML() failed',
>>> conn=self)
>>> libvirtError: internal error: process exited while connecting to monitor:
>>> 2016-12-20T21:54:43.044971Z qemu-kvm: warning: CPU(s) not present in any
>>> NUMA nodes: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>>> 2016-12-20T21:54:43.045164Z qemu-kvm: warning: All CPU(s) up to maxcpus
>>> should be described in NUMA config
>>> 2016-12-20T21:54:43.101886Z qemu-kvm: -device
>>> usb-ccid,id=ccid0,bus=usb.0,port=1: Warning: speed mismatch trying to
>>> attach usb device "QEMU USB CCID" (full speed) to bus "usb.0", port "1"
>>> (high speed)
>>
>> it is likely related to the recent USB patches
>> investigating
>
> hm, there are multiple problems (features/bugs depending on prefferred point
> of view:)
> but there is an easy “fix” taking care of this particular problem, so we can
> start with that and figure out the proper approach later
> arik will push that and merge it soon, likely today
>
> Thanks - if there is a quicker way to resolve this by reverting, I think it's
> a better option.
I really need to talk you out of this approach:-)
It does sound tempting and logical, but with our development model of large
patch series combined with late detection it really is quite risky. Here it
wouldn’t help much…and figuring out the right revert patch is more complicated
then fixing it.
I believe the best is to identify it early and notify the maintainer who merged
that patch ASAP, as that person is in the best position to asses if revert is
safe or if there is a simple follow up patch he can push right away
We can surely improve on reporting, so Barak, how/why did you point to that
particular patch in your email? It should start failing on
16c2ec236184b3152f1df8e874b43115f78d0989 (CommitDate: Fri Dec 16 01:56:07 2016
-0500)
Even though it may be that it was hidden because of
c46f653a7846c3c2a76507b8dcf5bc0391ec5709 (CommitDate: Mon Dec 19 15:16:40 2016
-0500)
(fix is ready, waiting on CI now)
Thanks,
michal
> Y.
>
>
>>> 2016-12-20 16:54:43,550 INFO (vm/d299ab29) [virt.vm]
>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') Changed state to Down:
>>> internal error: process exited while connecting to monitor:
>>> 2016-12-20T21:54:43.044971Z qemu-kvm: warning: CPU(s) not present in any
>>> NUMA nodes: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>>> 2016-12-20T21:54:43.045164Z qemu-kvm: warning: All CPU(s) up to maxcpus
>>> should be described in NUMA config
>>> 2016-12-20T21:54:43.101886Z qemu-kvm: -device
>>> usb-ccid,id=ccid0,bus=usb.0,port=1: Warning: speed mismatch trying to
>>> attach usb device "QEMU USB CCID" (full speed) to bus "usb.0", port "1"
>>> (high speed) (code=1) (vm:1197)
>>> 2016-12-20 16:54:43,550 INFO (vm/d299ab29) [virt.vm]
>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') Stopping connection
>>> (guestagent:430)
>>>
>>>
>>> And on The engine loads of these:
>>>
>>> 2016-12-20 16:53:57,844-05 ERROR
>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Command
>>> 'PollVDSCommand(HostName = lago-basic-suite-4-1-host0,
>>> VdsIdVDSCommandParametersBase:{runAsync='true',
>>> hostId='994b5d79-605f-4415-94f2-02c79cfa246e'})' execution failed:
>>> VDSGenericException: VDSNetworkException: Timeout during rpc call
>>> 2016-12-20 16:53:57,849-05 DEBUG
>>> [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp
>>> Reactor) [7971dfb4] MESSAGE
>>> content-length:80
>>> destination:jms.topic.vdsm_responses
>>> content-type:application/json
>>> subscription:5b6494d5-d5a0-4771-941c-a8be70f72450
>>>
>>> {"jsonrpc": "2.0", "id": "3c95fdb0-5b77-4927-9f6e-adc7395c122d", "result":
>>> true}�
>>> 2016-12-20 16:53:57,850-05 DEBUG
>>> [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) []
>>> Message received: {"jsonrpc": "2.0", "id":
>>> "3c95fdb0-5b77-4927-9f6e-adc7395c122d", "result": true}
>>> 2016-12-20 16:53:57,850-05 ERROR
>>> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
>>> to update response for "3c95fdb0-5b77-4927-9f6e-adc7395c122d"
>>> 2016-12-20 16:53:57,844-05 DEBUG
>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Exception:
>>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
>>> VDSGenericException: VDSNetworkException: Timeout during rpc call
>>> at
>>> org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:73)
>>> [vdsbroker.jar:]
>>> at
>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.getValue(HostSetupNetworkPoller.java:56)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(HostSetupNetworkPoller.java:41)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.invokeSetupNetworksCommand(HostSetupNetworksCommand.java:426)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.executeCommand(HostSetupNetworksCommand.java:287)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1249)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1389)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2053)
>>> [bll.jar:]
>>> at org.ovirt.engine.core.utils.tr
>>> <http://org.ovirt.engine.core.utils.tr/>ansaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164)
>>> [utils.jar:]
>>> at org.ovirt.engine.core.utils.tr
>>> <http://org.ovirt.engine.core.utils.tr/>ansaction.TransactionSupport.executeInScope(TransactionSupport.java:103)
>>> [utils.jar:]
>>> at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1449)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:395)
>>> [bll.jar:]
>>> at
>>> org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
>>> [bll.jar:]
>>> at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:511)
>>> [bll.jar:]
>>> at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:493)
>>> [bll.jar:]
>>> at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:446)
>>> [bll.jar:]
>>> at sun.reflect.GeneratedMethodAccessor232.invoke(Unknown Source)
>>> [:1.8.0_111]
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> [rt.jar:1.8.0_111]
>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>> at
>>> org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>> at
>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70)
>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80)
>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)
>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>> at
>>> org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13)
>>> [bll.jar:]
>>> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
>>> [:1.8.0_111]
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> [rt.jar:1.8.0_111]
>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>> at
>>> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.in
>>> <http://org.jboss.as.ejb3.component.in/>vocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>> at
>>> org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
>>> at
>>> org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
>>> [wildfly-ee-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>> at
>>> org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.si
>>> <http://org.jboss.as.ejb3.component.si/>ngleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:263)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.in
>>> <http://org.jboss.as.ejb3.component.in/>terceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.in
>>> <http://org.jboss.as.ejb3.component.in/>vocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.in
>>> <http://org.jboss.as.ejb3.component.in/>terceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ejb3.component.in
>>> <http://org.jboss.as.ejb3.component.in/>terceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at org.jboss.as.ee.component.Name
>>> <http://org.jboss.as.ee.component.name/>spaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
>>> at
>>> org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
>>> at
>>> org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
>>> at
>>> org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>> at
>>> org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
>>> at
>>> org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
>>> at
>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>> at
>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>> at
>>> org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
>>> at
>>> org.ovirt.engine.core.common.interfaces.BackendLocal$$$view4.runAction(Unknown
>>> Source) [common.jar:]
>>> at
>>> org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250)
>>> at
>>> org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:182)
>>> at
>>> org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:170)
>>> at org.ovirt.engine.api.restapi.resource.BackendHostResource.se
>>> <http://esource.backendhostresource.se/>tupNetworks(BackendHostResource.java:212)
>>> at org.ovirt.engine.api.v3.V3Server.adaptAction(V3Server.java:216)
>>> at
>>> org.ovirt.engine.api.v3.servers.V3HostServer.setupNetworks(V3HostServer.java:182)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> [rt.jar:1.8.0_111]
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> [rt.jar:1.8.0_111]
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> [rt.jar:1.8.0_111]
>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>> at
>>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at
>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>> [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
>>> at
>>> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81)
>>> at
>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>> at
>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209)
>>> at
>>> io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221)
>>> at
>>> io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147)
>>> at
>>> io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111)
>>> at
>>> org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:139)
>>> at
>>> org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:68)
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:116)
>>> at
>>> org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:71)
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:91)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.do
>>> <http://ers.sessionvalidationfilter.do/>Filter(SessionValidationFilter.java:59)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35)
>>> [aaa.jar:]
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)
>>> at
>>> org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> org.ovirt.engine.api.restapi.security.CORSSupportFilter.doFilter(CORSSupportFilter.java:183)
>>> at
>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> at
>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
>>> at
>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>> at
>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>> at
>>> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at
>>> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
>>> at
>>> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at
>>> io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
>>> at
>>> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
>>> at
>>> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
>>> at
>>> io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
>>> at
>>> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
>>> at
>>> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
>>> at
>>> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
>>> at
>>> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at
>>> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at io.undertow.server.handlers.Pr
>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
>>> at
>>> io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
>>> at
>>> io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
>>> at
>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>> at
>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>> at
>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>> at
>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
>>> at
>>> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
>>> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
>>> at
>>> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [rt.jar:1.8.0_111]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [rt.jar:1.8.0_111]
>>> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_111]
>>>
>>> 2016-12-20 16:53:57,859-05 ERROR
>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Timeout waiting for VDSM
>>> response: Internal timeout occured
>>>
>>> This is definitely not related to the patch linked above.
>>>
>>> However I am also not quite sure what is the cause of this right now, it
>>> might be a breaking change in VDSM.
>>>
>>>
>>> The DC level is 4.0, is it possible that is the reason for the failure?
>>>
>>>
>>>
>>>>
>>>> Full test code can be seen here:
>>>> https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=basic-suite-4.1/test-scenarios/004_basic_sanity.py;hb=refs/heads/master#l291
>>>>
>>>> <https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=basic-suite-4.1/test-scenarios/004_basic_sanity.py;hb=refs/heads/master#l291>
>>>>
>>>> Full test exception can be seen here:
>>>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/testReport/junit/%28root%29/004_basic_sanity/vm_run/
>>>>
>>>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/testReport/junit/%28root%29/004_basic_sanity/vm_run/>
>>>>
>>>> Further logs can be seen in Jenkins:
>>>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/artifact/exported-artifacts/basic_suite_4.1.sh-el7/exported-artifacts/test_logs/basic-suite-4.1/post-004_basic_sanity.py/
>>>>
>>>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/artifact/exported-artifacts/basic_suite_4.1.sh-el7/exported-artifacts/test_logs/basic-suite-4.1/post-004_basic_sanity.py/>
>>>>
>>>> --
>>>> Barak Korren
>>>> [email protected] <mailto:[email protected]>
>>>> RHCE, RHCi, RHV-DevOps Team
>>>> https://ifireball.wordpress.com/ <https://ifireball.wordpress.com/>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> [email protected] <mailto:[email protected]>
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>>
>>>
>>>
>>> --
>>> Eyal Edri
>>> Associate Manager
>>> RHV DevOps
>>> EMEA ENG Virtualization R&D
>>> Red Hat Israel
>>>
>>> phone: +972-9-7692018 <tel:+972%209-769-2018>
>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>> _______________________________________________
>>> Devel mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>
>
> _______________________________________________
> Devel mailing list
> [email protected] <mailto:[email protected]>
> http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
>
_______________________________________________
Devel mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/devel
