> On 21 Dec 2016, at 16:41, Eyal Edri <[email protected]> wrote:
>
>
>
> On Wed, Dec 21, 2016 at 5:36 PM, Michal Skrivanek
> <[email protected] <mailto:[email protected]>> wrote:
>
>> On 21 Dec 2016, at 16:25, Yaniv Kaul <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>
>>
>> On Wed, Dec 21, 2016 at 5:19 PM, Michal Skrivanek
>> <[email protected] <mailto:[email protected]>> wrote:
>>
>>> On 21 Dec 2016, at 14:56, Michal Skrivanek <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>
>>>> On 21 Dec 2016, at 12:19, Eyal Edri <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>>
>>>>
>>>> On Wed, Dec 21, 2016 at 12:56 PM, Vinzenz Feenstra <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>>> On Dec 21, 2016, at 11:17 AM, Barak Korren <[email protected]
>>>>> <mailto:[email protected]>> wrote:
>>>>>
>>>>> The test for running VMs had been failing since yesterday.
>>>>>
>>>>> The patch merged before the failures started was:
>>>>> https://gerrit.ovirt.org/#/c/68826/ <https://gerrit.ovirt.org/#/c/68826/>
>>>>
>>>>
>>>>
>>>>>
>>>>> The error we`re seeing is a time-out (after two minutes) while running
>>>>> this API call:
>>>>>
>>>>> api.vms.get(VM0_NAME).status.state == ‘up'
>>>>
>>>> This is a REST API call, the patch above is Frontend. So this is unrelated.
>>>>
>>>> However on Host 0 I can see this:
>>>>
>>>> 2016-12-20 16:54:43,544 ERROR (vm/d299ab29) [virt.vm]
>>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') The vm start process failed
>>>> (vm:615)
>>>> Traceback (most recent call last):
>>>> File "/usr/share/vdsm/virt/vm.py", line 551, in _startUnderlyingVm
>>>> self._run()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1991, in _run
>>>> self._connection.createXML(domxml, flags),
>>>> File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
>>>> 123, in wrapper
>>>> ret = f(*args, **kwargs)
>>>> File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 941, in
>>>> wrapper
>>>> return func(inst, *args, **kwargs)
>>>> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3782, in
>>>> createXML
>>>> if ret is None:raise libvirtError('virDomainCreateXML() failed',
>>>> conn=self)
>>>> libvirtError: internal error: process exited while connecting to monitor:
>>>> 2016-12-20T21:54:43.044971Z qemu-kvm: warning: CPU(s) not present in any
>>>> NUMA nodes: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>>>> 2016-12-20T21:54:43.045164Z qemu-kvm: warning: All CPU(s) up to maxcpus
>>>> should be described in NUMA config
>>>> 2016-12-20T21:54:43.101886Z qemu-kvm: -device
>>>> usb-ccid,id=ccid0,bus=usb.0,port=1: Warning: speed mismatch trying to
>>>> attach usb device "QEMU USB CCID" (full speed) to bus "usb.0", port "1"
>>>> (high speed)
>>>
>>> it is likely related to the recent USB patches
>>> investigating
>>
>> hm, there are multiple problems (features/bugs depending on prefferred point
>> of view:)
>> but there is an easy “fix” taking care of this particular problem, so we can
>> start with that and figure out the proper approach later
>> arik will push that and merge it soon, likely today
>>
>> Thanks - if there is a quicker way to resolve this by reverting, I think
>> it's a better option.
>
> I really need to talk you out of this approach:-)
> It does sound tempting and logical, but with our development model of large
> patch series combined with late detection it really is quite risky. Here it
> wouldn’t help much…and figuring out the right revert patch is more
> complicated then fixing it.
>
> Can we start asking developers run OST before they merge so it will be early
> detection and not late detection?
> We have video sessions on how to use OST, so it shouldn't be any issues in
> running it on a patch.
why do you think i’m so annoying on those dependency issues:-)
it is not currently possible for most people. Different reasons.
plus all the various recent breakages in past 2 weeks made it even worse:/
>
> I believe the best is to identify it early and notify the maintainer who
> merged that patch ASAP, as that person is in the best position to asses if
> revert is safe or if there is a simple follow up patch he can push right away
>
> We can surely improve on reporting, so Barak, how/why did you point to that
> particular patch in your email? It should start failing on
> 16c2ec236184b3152f1df8e874b43115f78d0989 (CommitDate: Fri Dec 16 01:56:07
> 2016 -0500)
> Even though it may be that it was hidden because of
> c46f653a7846c3c2a76507b8dcf5bc0391ec5709 (CommitDate: Mon Dec 19 15:16:40
> 2016 -0500)
>
> (fix is ready, waiting on CI now)
>
> Thanks,
> michal
>
>> Y.
>>
>>
>>>> 2016-12-20 16:54:43,550 INFO (vm/d299ab29) [virt.vm]
>>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') Changed state to Down:
>>>> internal error: process exited while connecting to monitor:
>>>> 2016-12-20T21:54:43.044971Z qemu-kvm: warning: CPU(s) not present in any
>>>> NUMA nodes: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>>>> 2016-12-20T21:54:43.045164Z qemu-kvm: warning: All CPU(s) up to maxcpus
>>>> should be described in NUMA config
>>>> 2016-12-20T21:54:43.101886Z qemu-kvm: -device
>>>> usb-ccid,id=ccid0,bus=usb.0,port=1: Warning: speed mismatch trying to
>>>> attach usb device "QEMU USB CCID" (full speed) to bus "usb.0", port "1"
>>>> (high speed) (code=1) (vm:1197)
>>>> 2016-12-20 16:54:43,550 INFO (vm/d299ab29) [virt.vm]
>>>> (vmId='d299ab29-284a-435c-a50f-183a6e54def2') Stopping connection
>>>> (guestagent:430)
>>>>
>>>>
>>>> And on The engine loads of these:
>>>>
>>>> 2016-12-20 16:53:57,844-05 ERROR
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Command
>>>> 'PollVDSCommand(HostName = lago-basic-suite-4-1-host0,
>>>> VdsIdVDSCommandParametersBase:{runAsync='true',
>>>> hostId='994b5d79-605f-4415-94f2-02c79cfa246e'})' execution failed:
>>>> VDSGenericException: VDSNetworkException: Timeout during rpc call
>>>> 2016-12-20 16:53:57,849-05 DEBUG
>>>> [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp
>>>> Reactor) [7971dfb4] MESSAGE
>>>> content-length:80
>>>> destination:jms.topic.vdsm_responses
>>>> content-type:application/json
>>>> subscription:5b6494d5-d5a0-4771-941c-a8be70f72450
>>>>
>>>> {"jsonrpc": "2.0", "id": "3c95fdb0-5b77-4927-9f6e-adc7395c122d", "result":
>>>> true}�
>>>> 2016-12-20 16:53:57,850-05 DEBUG
>>>> [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker)
>>>> [] Message received: {"jsonrpc": "2.0", "id":
>>>> "3c95fdb0-5b77-4927-9f6e-adc7395c122d", "result": true}
>>>> 2016-12-20 16:53:57,850-05 ERROR
>>>> [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able
>>>> to update response for "3c95fdb0-5b77-4927-9f6e-adc7395c122d"
>>>> 2016-12-20 16:53:57,844-05 DEBUG
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Exception:
>>>> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
>>>> VDSGenericException: VDSNetworkException: Timeout during rpc call
>>>> at
>>>> org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:73)
>>>> [vdsbroker.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.getValue(HostSetupNetworkPoller.java:56)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(HostSetupNetworkPoller.java:41)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.invokeSetupNetworksCommand(HostSetupNetworksCommand.java:426)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.executeCommand(HostSetupNetworksCommand.java:287)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1249)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1389)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2053)
>>>> [bll.jar:]
>>>> at org.ovirt.engine.core.utils.tr
>>>> <http://org.ovirt.engine.core.utils.tr/>ansaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164)
>>>> [utils.jar:]
>>>> at org.ovirt.engine.core.utils.tr
>>>> <http://org.ovirt.engine.core.utils.tr/>ansaction.TransactionSupport.executeInScope(TransactionSupport.java:103)
>>>> [utils.jar:]
>>>> at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1449)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:395)
>>>> [bll.jar:]
>>>> at
>>>> org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
>>>> [bll.jar:]
>>>> at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:511)
>>>> [bll.jar:]
>>>> at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:493)
>>>> [bll.jar:]
>>>> at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:446)
>>>> [bll.jar:]
>>>> at sun.reflect.GeneratedMethodAccessor232.invoke(Unknown Source)
>>>> [:1.8.0_111]
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> [rt.jar:1.8.0_111]
>>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>>> at
>>>> org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>>> at
>>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70)
>>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80)
>>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)
>>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>>> at
>>>> org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13)
>>>> [bll.jar:]
>>>> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
>>>> [:1.8.0_111]
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> [rt.jar:1.8.0_111]
>>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>>> at
>>>> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.in
>>>> <http://org.jboss.as.ejb3.component.in/>vocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
>>>> at
>>>> org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
>>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
>>>> at
>>>> org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
>>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
>>>> [wildfly-ee-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>>> at
>>>> org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.si
>>>> <http://org.jboss.as.ejb3.component.si/>ngleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:263)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.in
>>>> <http://org.jboss.as.ejb3.component.in/>terceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.in
>>>> <http://org.jboss.as.ejb3.component.in/>vocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.in
>>>> <http://org.jboss.as.ejb3.component.in/>terceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ejb3.component.in
>>>> <http://org.jboss.as.ejb3.component.in/>terceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
>>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at org.jboss.as.ee.component.Name
>>>> <http://org.jboss.as.ee.component.name/>spaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
>>>> at
>>>> org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
>>>> at
>>>> org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
>>>> at
>>>> org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>>> at
>>>> org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
>>>> at
>>>> org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
>>>> at
>>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
>>>> at
>>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>>>> at
>>>> org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
>>>> at
>>>> org.ovirt.engine.core.common.interfaces.BackendLocal$$$view4.runAction(Unknown
>>>> Source) [common.jar:]
>>>> at
>>>> org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250)
>>>> at
>>>> org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:182)
>>>> at
>>>> org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:170)
>>>> at org.ovirt.engine.api.restapi.resource.BackendHostResource.se
>>>> <http://esource.backendhostresource.se/>tupNetworks(BackendHostResource.java:212)
>>>> at org.ovirt.engine.api.v3.V3Server.adaptAction(V3Server.java:216)
>>>> at
>>>> org.ovirt.engine.api.v3.servers.V3HostServer.setupNetworks(V3HostServer.java:182)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> [rt.jar:1.8.0_111]
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> [rt.jar:1.8.0_111]
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> [rt.jar:1.8.0_111]
>>>> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_111]
>>>> at
>>>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at
>>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>>>> [resteasy-jaxrs-3.0.19.Final.jar:3.0.19.Final]
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>>> [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
>>>> at
>>>> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81)
>>>> at
>>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>>> at
>>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209)
>>>> at
>>>> io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221)
>>>> at
>>>> io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147)
>>>> at
>>>> io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111)
>>>> at
>>>> org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:139)
>>>> at
>>>> org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:68)
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:116)
>>>> at
>>>> org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:71)
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:91)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.do
>>>> <http://ers.sessionvalidationfilter.do/>Filter(SessionValidationFilter.java:59)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35)
>>>> [aaa.jar:]
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)
>>>> at
>>>> org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> org.ovirt.engine.api.restapi.security.CORSSupportFilter.doFilter(CORSSupportFilter.java:183)
>>>> at
>>>> io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>>> at
>>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
>>>> at
>>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>>> at
>>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>>> at
>>>> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at
>>>> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
>>>> at
>>>> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at
>>>> io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
>>>> at
>>>> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
>>>> at
>>>> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
>>>> at
>>>> io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
>>>> at
>>>> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
>>>> at
>>>> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
>>>> at
>>>> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
>>>> at
>>>> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at
>>>> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at io.undertow.server.handlers.Pr
>>>> <http://io.undertow.server.handlers.pr/>edicateHandler.handleRequest(PredicateHandler.java:43)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
>>>> at
>>>> io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
>>>> at
>>>> io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
>>>> at
>>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>>> at
>>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>>> at
>>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>>> at
>>>> io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
>>>> at
>>>> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
>>>> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
>>>> at
>>>> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> [rt.jar:1.8.0_111]
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> [rt.jar:1.8.0_111]
>>>> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_111]
>>>>
>>>> 2016-12-20 16:53:57,859-05 ERROR
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default
>>>> task-17) [5ecd5a55-2b7a-4dd6-b42b-cc49bbfb3962] Timeout waiting for VDSM
>>>> response: Internal timeout occured
>>>>
>>>> This is definitely not related to the patch linked above.
>>>>
>>>> However I am also not quite sure what is the cause of this right now, it
>>>> might be a breaking change in VDSM.
>>>>
>>>>
>>>> The DC level is 4.0, is it possible that is the reason for the failure?
>>>>
>>>>
>>>>
>>>>>
>>>>> Full test code can be seen here:
>>>>> https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=basic-suite-4.1/test-scenarios/004_basic_sanity.py;hb=refs/heads/master#l291
>>>>>
>>>>> <https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=basic-suite-4.1/test-scenarios/004_basic_sanity.py;hb=refs/heads/master#l291>
>>>>>
>>>>> Full test exception can be seen here:
>>>>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/testReport/junit/%28root%29/004_basic_sanity/vm_run/
>>>>>
>>>>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/testReport/junit/%28root%29/004_basic_sanity/vm_run/>
>>>>>
>>>>> Further logs can be seen in Jenkins:
>>>>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/artifact/exported-artifacts/basic_suite_4.1.sh-el7/exported-artifacts/test_logs/basic-suite-4.1/post-004_basic_sanity.py/
>>>>>
>>>>> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/13/artifact/exported-artifacts/basic_suite_4.1.sh-el7/exported-artifacts/test_logs/basic-suite-4.1/post-004_basic_sanity.py/>
>>>>>
>>>>> --
>>>>> Barak Korren
>>>>> [email protected] <mailto:[email protected]>
>>>>> RHCE, RHCi, RHV-DevOps Team
>>>>> https://ifireball.wordpress.com/ <https://ifireball.wordpress.com/>
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> [email protected] <mailto:[email protected]>
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> [email protected] <mailto:[email protected]>
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>>>
>>>>
>>>>
>>>> --
>>>> Eyal Edri
>>>> Associate Manager
>>>> RHV DevOps
>>>> EMEA ENG Virtualization R&D
>>>> Red Hat Israel
>>>>
>>>> phone: +972-9-7692018 <tel:+972%209-769-2018>
>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>> _______________________________________________
>>>> Devel mailing list
>>>> [email protected] <mailto:[email protected]>
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.ovirt.org/mailman/listinfo/devel
>> <http://lists.ovirt.org/mailman/listinfo/devel>
>>
>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________
Devel mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/devel
