date:20161129

Re: [ovirt-devel] VDSM changes Linux memory dirty ratios - why?

2016-11-29 Thread Martin Polednik


On 29/11/16 22:01 +0200, Yaniv Kaul wrote:

It appears that VDSM changes the following params:
vm.dirty_ratio = 5
vm.dirty_background_ratio = 2

Any idea why? Because we use cache=none it's irrelevant anyway?


It's not really irrelevant, the host still uses disk cache. Anyway,
there is BZ[1] with a presentation[2] that (imho reasonably) states:

"Reduce dirty page limits in KVM host to allow
direct I/O writer VMs to compete successfully
with buffered writer processes for storage
access"

I wonder why virtual-host tuned profile doesn't contain these values:

$ grep vm.dirty /usr/lib/tuned/virtual-host/tuned.conf
vm.dirty_background_ratio = 5

[1]https://bugzilla.redhat.com/show_bug.cgi?id=740887
[2]http://perf1.lab.bos.redhat.com/bengland/laptop/rhev/rhev-vm-rsptime.pdf


TIA,
Y.



___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't start

2016-11-29 Thread Martin Perina

Hi,

here's the patch that should fix the issue:

https://gerrit.ovirt.org/67559

Martin


On Wed, Nov 30, 2016 at 6:52 AM, Martin Perina  wrote:

> Hi,
>
> I will post a fix in a few minutes, sorry for the breakage.
>
> Martin
>
>
> On Wed, Nov 30, 2016 at 4:53 AM, Greg Sheremeta 
> wrote:
>
>> Reverting 6d6c6bdc fixes the issue.
>>
>> core: xmlrpc removal
>>
>> Now with jsonrpc being primary protocol we fade out xmlrpc
>> implementation.
>>
>> @Piotr, please advise.
>>
>> Best wishes,
>> Greg
>>
>>
>>
>>
>>
>> On Tue, Nov 29, 2016 at 9:44 PM, Greg Sheremeta 
>> wrote:
>>
>>> Hi,
>>>
>>> A fresh checkout of master with a fresh database will not start for me.
>>> I tried on two separate machines. ovirt-engine-4.0 branch is still working.
>>>
>>> Any ideas?
>>>
>>> Best wishes,
>>> Greg
>>>
>>>
>>> 2016-11-29 21:11:33,918-05 ERROR [org.jboss.msc.service.fail] (MSC
>>> service thread 1-4) MSC01: Failed to start service
>>> jboss.module.service."deployment.engine.ear.docs.war".main:
>>> org.jboss.msc.service.StartException in service
>>> jboss.module.service."deployment.engine.ear.docs.war".main:
>>> WFLYSRV0179: Failed to load module: deployment.engine.ear.docs.war:main
>>> at 
>>> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
>>> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
>>> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startS
>>> ervice(ServiceControllerImpl.java:1948) [jboss-msc-1.2.6.Final.jar:1.2
>>> .6.Final]
>>> at 
>>> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
>>> [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [rt.jar:1.8.0_101]
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [rt.jar:1.8.0_101]
>>> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_101]
>>> Caused by: org.jboss.modules.ModuleLoadException: Error loading module
>>> from /home/greg/ovirt-engine/share/ovirt-engine/modules/common/or
>>> g/apache/xmlrpc/main/module.xml
>>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>>> mlParser.java:228)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>>> mlParser.java:204)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>>> mlParser.java:170)
>>> at 
>>> org.jboss.modules.LocalModuleFinder.lambda$findModule$3(LocalModuleFinder.java:149)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> [rt.jar:1.8.0_101]
>>> at 
>>> org.jboss.modules.LocalModuleFinder.findModule(LocalModuleFinder.java:144)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.ModuleLoader.findModule(ModuleLoader.java:439)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:342)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:289)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at 
>>> org.jboss.modules.ModuleLoader.preloadExportedModule(ModuleLoader.java:300)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:313)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.as.server.moduleservice.ServiceModuleLoader.preloa
>>> dModule(ServiceModuleLoader.java:149) [wildfly-server-2.0.10.Final.j
>>> ar:2.0.10.Final]
>>> at org.jboss.modules.Module.addExportedPaths(Module.java:1229)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.Module.addPaths(Module.java:1121)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.Module.link(Module.java:1448)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.Module.relinkIfNecessary(Module.java:1476)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:225)
>>> [jboss-modules.jar:1.5.1.Final]
>>> at 
>>> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:68)
>>> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
>>> ... 5 more
>>> Caused by: org.jboss.modules.xml.XmlPullParserException: Failed to add
>>> resource root 'xmlrpc-common.jar' at path 'xmlrpc-common.jar' (position:
>>> END_TAG seen ...\n>> path="xmlrpc-common.jar"/>... @6:46) caused by:
>>> java.io.FileNotFoundException: /home/greg/ovirt-engine/share/
>>> ovirt-engine/modules/common/org/apache/xmlrpc/main/xmlrpc-common.jar
>>> (No such file or directory)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseResourceRoot(Modu
>>> leXmlParser.java:891)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseResources(ModuleX
>>> mlParser.java:735)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleContents(Mo
>>> duleXmlParser.java:535)
>>> at org.jboss.modules.xml.ModuleXmlParser.parseDocument(ModuleXm
>>> lParser.java:340)
>>> at

Re: [ovirt-devel] master won't start

2016-11-29 Thread Martin Perina

Hi,

I will post a fix in a few minutes, sorry for the breakage.

Martin


On Wed, Nov 30, 2016 at 4:53 AM, Greg Sheremeta  wrote:

> Reverting 6d6c6bdc fixes the issue.
>
> core: xmlrpc removal
>
> Now with jsonrpc being primary protocol we fade out xmlrpc
> implementation.
>
> @Piotr, please advise.
>
> Best wishes,
> Greg
>
>
>
>
>
> On Tue, Nov 29, 2016 at 9:44 PM, Greg Sheremeta 
> wrote:
>
>> Hi,
>>
>> A fresh checkout of master with a fresh database will not start for me. I
>> tried on two separate machines. ovirt-engine-4.0 branch is still working.
>>
>> Any ideas?
>>
>> Best wishes,
>> Greg
>>
>>
>> 2016-11-29 21:11:33,918-05 ERROR [org.jboss.msc.service.fail] (MSC
>> service thread 1-4) MSC01: Failed to start service
>> jboss.module.service."deployment.engine.ear.docs.war".main:
>> org.jboss.msc.service.StartException in service
>> jboss.module.service."deployment.engine.ear.docs.war".main: WFLYSRV0179:
>> Failed to load module: deployment.engine.ear.docs.war:main
>> at 
>> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
>> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
>> at org.jboss.msc.service.ServiceControllerImpl$StartTask.
>> startService(ServiceControllerImpl.java:1948)
>> [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
>> at 
>> org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
>> [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> [rt.jar:1.8.0_101]
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [rt.jar:1.8.0_101]
>> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_101]
>> Caused by: org.jboss.modules.ModuleLoadException: Error loading module
>> from /home/greg/ovirt-engine/share/ovirt-engine/modules/common/or
>> g/apache/xmlrpc/main/module.xml
>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>> mlParser.java:228)
>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>> mlParser.java:204)
>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>> mlParser.java:170)
>> at 
>> org.jboss.modules.LocalModuleFinder.lambda$findModule$3(LocalModuleFinder.java:149)
>> [jboss-modules.jar:1.5.1.Final]
>> at java.security.AccessController.doPrivileged(Native Method)
>> [rt.jar:1.8.0_101]
>> at org.jboss.modules.LocalModuleFinder.findModule(LocalModuleFinder.java:144)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.ModuleLoader.findModule(ModuleLoader.java:439)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:342)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:289)
>> [jboss-modules.jar:1.5.1.Final]
>> at 
>> org.jboss.modules.ModuleLoader.preloadExportedModule(ModuleLoader.java:300)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:313)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.as.server.moduleservice.ServiceModuleLoader.preloa
>> dModule(ServiceModuleLoader.java:149) [wildfly-server-2.0.10.Final.j
>> ar:2.0.10.Final]
>> at org.jboss.modules.Module.addExportedPaths(Module.java:1229)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.Module.addPaths(Module.java:1121)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.Module.link(Module.java:1448)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.Module.relinkIfNecessary(Module.java:1476)
>> [jboss-modules.jar:1.5.1.Final]
>> at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:225)
>> [jboss-modules.jar:1.5.1.Final]
>> at 
>> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:68)
>> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
>> ... 5 more
>> Caused by: org.jboss.modules.xml.XmlPullParserException: Failed to add
>> resource root 'xmlrpc-common.jar' at path 'xmlrpc-common.jar' (position:
>> END_TAG seen ...\n> path="xmlrpc-common.jar"/>... @6:46) caused by:
>> java.io.FileNotFoundException: /home/greg/ovirt-engine/share/
>> ovirt-engine/modules/common/org/apache/xmlrpc/main/xmlrpc-common.jar (No
>> such file or directory)
>> at org.jboss.modules.xml.ModuleXmlParser.parseResourceRoot(Modu
>> leXmlParser.java:891)
>> at org.jboss.modules.xml.ModuleXmlParser.parseResources(ModuleX
>> mlParser.java:735)
>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleContents(Mo
>> duleXmlParser.java:535)
>> at org.jboss.modules.xml.ModuleXmlParser.parseDocument(ModuleXm
>> lParser.java:340)
>> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleX
>> mlParser.java:226)
>> ... 22 more
>>
>> 2016-11-29 21:11:33,915-05 ERROR [org.jboss.msc.service.fail] (MSC
>> service thread 1-5) MSC01: Failed to start service
>> jboss.module.service."deployment.engine.ear.welcome.war".main:
>>

Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

2016-11-29 Thread Ramesh Nachimuthu





- Original Message -
> From: "Nir Soffer" 
> To: "Ramesh Nachimuthu" 
> Cc: "devel" , "Dan Kenigsberg" , "Piotr 
> Kliczewski" , "Yaniv
> Bronheim" , "Aravinda Vishwanathapura Krishna Murthy" 
> , "Sahina Bose"
> 
> Sent: Tuesday, November 29, 2016 7:20:57 PM
> Subject: Re: python path conflicts issues while running python scripts from 
> vdsm verbs
> 
> On Tue, Nov 29, 2016 at 3:26 PM, Ramesh Nachimuthu 
> wrote:
> > Hi,
> >
> >  I am trying to run a python script '/sbin/gluster-eventsapi' in vdsm verb
> >  which internally imports some python modules from
> >  /usr/lib/python2.7/site-packages/gluster/cliutils/. But it fails with the
> >  import error. Following error is seen in the supervdsm log.
> >
> >
> > MainProcess|Thread-1::DEBUG::2016-11-28
> > 16:54:35,130::commands::93::root::(execCmd) FAILED:  = 'Traceback
> > (most recent call last):\n  File "/sbin/gluster-eventsapi", line 25, in
> > \nfrom gluster.cliutils.cliutils import (Cmd, execute,
> > node_output_ok, node_output_notok,\nImportError: No module named
> > cliutils.cliutils\n';  = 1
> >
> >
> > I think the import statement "from gluster.cliutils.cliutils import (Cmd,
> > execute, node_output_ok, node_output_notok)" in the python script resolves
> > to '/usr/share/vdsm/gluster' instead of
> > /usr/lib/python2.7/site-packages/gluster/cliutils.
> >
> > I see the following in python sys.path while executing a python script from
> > vdsm.
> >
> > ['/usr/libexec/glusterfs', '/usr/share/vdsm', '/usr/lib64/python27.zip',
> > '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2',
> > '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old',
> > '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages',
> > '/usr/lib64/python2.7/site-packages/gtk-2.0',
> > '/usr/lib/python2.7/site-packages']
> >
> > Looks like '/usr/share/vdsm' take precedence over
> > '/usr/lib64/python2.7/site-packages'.
> >
> > Can someone suggests a way to fix this issue?
> >
> > Note: '/sbin/gluster-eventsapi' works perfectly while running directly from
> > CLI.
> >
> > Related vdsm patch:
> > https://gerrit.ovirt.org/#/c/67168/2/vdsm/gluster/events.py
> 
> If an import fails because /usr/share/vdsm/foo hides a package in
> /usr/lib/python2.7/site-packages/vdsm it means that you have a wrong
> import, you need to import anything like this:
> 
> from vdsm.foo import bar
> 
> So names in /usr/share/vdsm/* cannot hide names in vdsm package.
> 

Here its a different case. Some external python script (which I am trying to 
run from vdsm verb) is trying to import some gluster package from 
/usr/lib/python2.7/site-packages/gluster. But it is conflicting with gluster 
modules under /usr/share/vdsm/gluster

Python Script which I am trying to run: 
https://github.com/gluster/glusterfs/blob/master/events/src/peer_eventsapi.py

Regards,
Ramesh

> Nir
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] master won't start

2016-11-29 Thread Greg Sheremeta

Reverting 6d6c6bdc fixes the issue.

core: xmlrpc removal

Now with jsonrpc being primary protocol we fade out xmlrpc
implementation.

@Piotr, please advise.

Best wishes,
Greg





On Tue, Nov 29, 2016 at 9:44 PM, Greg Sheremeta  wrote:

> Hi,
>
> A fresh checkout of master with a fresh database will not start for me. I
> tried on two separate machines. ovirt-engine-4.0 branch is still working.
>
> Any ideas?
>
> Best wishes,
> Greg
>
>
> 2016-11-29 21:11:33,918-05 ERROR [org.jboss.msc.service.fail] (MSC service
> thread 1-4) MSC01: Failed to start service jboss.module.service."
> deployment.engine.ear.docs.war".main: org.jboss.msc.service.StartException
> in service jboss.module.service."deployment.engine.ear.docs.war".main:
> WFLYSRV0179: Failed to load module: deployment.engine.ear.docs.war:main
> at 
> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(
> ServiceControllerImpl.java:1948) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(
> ServiceControllerImpl.java:1881) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [rt.jar:1.8.0_101]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [rt.jar:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_101]
> Caused by: org.jboss.modules.ModuleLoadException: Error loading module
> from /home/greg/ovirt-engine/share/ovirt-engine/modules/common/
> org/apache/xmlrpc/main/module.xml
> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(
> ModuleXmlParser.java:228)
> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(
> ModuleXmlParser.java:204)
> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(
> ModuleXmlParser.java:170)
> at 
> org.jboss.modules.LocalModuleFinder.lambda$findModule$3(LocalModuleFinder.java:149)
> [jboss-modules.jar:1.5.1.Final]
> at java.security.AccessController.doPrivileged(Native Method)
> [rt.jar:1.8.0_101]
> at org.jboss.modules.LocalModuleFinder.findModule(LocalModuleFinder.java:144)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.findModule(ModuleLoader.java:439)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:342)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:289)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.preloadExportedModule(ModuleLoader.java:300)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:313)
> [jboss-modules.jar:1.5.1.Final]
> at 
> org.jboss.as.server.moduleservice.ServiceModuleLoader.preloadModule(ServiceModuleLoader.java:149)
> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
> at org.jboss.modules.Module.addExportedPaths(Module.java:1229)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.Module.addPaths(Module.java:1121)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.Module.link(Module.java:1448)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.Module.relinkIfNecessary(Module.java:1476)
> [jboss-modules.jar:1.5.1.Final]
> at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:225)
> [jboss-modules.jar:1.5.1.Final]
> at 
> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:68)
> [wildfly-server-2.0.10.Final.jar:2.0.10.Final]
> ... 5 more
> Caused by: org.jboss.modules.xml.XmlPullParserException: Failed to add
> resource root 'xmlrpc-common.jar' at path 'xmlrpc-common.jar' (position:
> END_TAG seen ...\n path="xmlrpc-common.jar"/>... @6:46) caused by:
> java.io.FileNotFoundException: /home/greg/ovirt-engine/share/
> ovirt-engine/modules/common/org/apache/xmlrpc/main/xmlrpc-common.jar (No
> such file or directory)
> at org.jboss.modules.xml.ModuleXmlParser.parseResourceRoot(
> ModuleXmlParser.java:891)
> at org.jboss.modules.xml.ModuleXmlParser.parseResources(
> ModuleXmlParser.java:735)
> at org.jboss.modules.xml.ModuleXmlParser.parseModuleContents(
> ModuleXmlParser.java:535)
> at org.jboss.modules.xml.ModuleXmlParser.parseDocument(
> ModuleXmlParser.java:340)
> at org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(
> ModuleXmlParser.java:226)
> ... 22 more
>
> 2016-11-29 21:11:33,915-05 ERROR [org.jboss.msc.service.fail] (MSC service
> thread 1-5) MSC01: Failed to start service jboss.module.service."
> deployment.engine.ear.welcome.war".main: org.jboss.msc.service.StartException
> in service jboss.module.service."deployment.engine.ear.welcome.war".main:
> WFLYSRV0179: Failed to load module: deployment.engine.ear.welcome.war:main
> at 
> org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
>

[ovirt-devel] master won't start

2016-11-29 Thread Greg Sheremeta

Hi,

A fresh checkout of master with a fresh database will not start for me. I
tried on two separate machines. ovirt-engine-4.0 branch is still working.

Any ideas?

Best wishes,
Greg


2016-11-29 21:11:33,918-05 ERROR [org.jboss.msc.service.fail] (MSC service
thread 1-4) MSC01: Failed to start service
jboss.module.service."deployment.engine.ear.docs.war".main:
org.jboss.msc.service.StartException in service
jboss.module.service."deployment.engine.ear.docs.war".main: WFLYSRV0179:
Failed to load module: deployment.engine.ear.docs.war:main
at
org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
[wildfly-server-2.0.10.Final.jar:2.0.10.Final]
at
org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
[jboss-msc-1.2.6.Final.jar:1.2.6.Final]
at
org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
[jboss-msc-1.2.6.Final.jar:1.2.6.Final]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[rt.jar:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[rt.jar:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_101]
Caused by: org.jboss.modules.ModuleLoadException: Error loading module from
/home/greg/ovirt-engine/share/ovirt-engine/modules/common/org/apache/xmlrpc/main/module.xml
at
org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleXmlParser.java:228)
at
org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleXmlParser.java:204)
at
org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleXmlParser.java:170)
at
org.jboss.modules.LocalModuleFinder.lambda$findModule$3(LocalModuleFinder.java:149)
[jboss-modules.jar:1.5.1.Final]
at java.security.AccessController.doPrivileged(Native Method)
[rt.jar:1.8.0_101]
at
org.jboss.modules.LocalModuleFinder.findModule(LocalModuleFinder.java:144)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.ModuleLoader.findModule(ModuleLoader.java:439)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:342)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:289)
[jboss-modules.jar:1.5.1.Final]
at
org.jboss.modules.ModuleLoader.preloadExportedModule(ModuleLoader.java:300)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:313)
[jboss-modules.jar:1.5.1.Final]
at
org.jboss.as.server.moduleservice.ServiceModuleLoader.preloadModule(ServiceModuleLoader.java:149)
[wildfly-server-2.0.10.Final.jar:2.0.10.Final]
at org.jboss.modules.Module.addExportedPaths(Module.java:1229)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.Module.addPaths(Module.java:1121)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.Module.link(Module.java:1448)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.Module.relinkIfNecessary(Module.java:1476)
[jboss-modules.jar:1.5.1.Final]
at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:225)
[jboss-modules.jar:1.5.1.Final]
at
org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:68)
[wildfly-server-2.0.10.Final.jar:2.0.10.Final]
... 5 more
Caused by: org.jboss.modules.xml.XmlPullParserException: Failed to add
resource root 'xmlrpc-common.jar' at path 'xmlrpc-common.jar' (position:
END_TAG seen ...\n... @6:46) caused by:
java.io.FileNotFoundException:
/home/greg/ovirt-engine/share/ovirt-engine/modules/common/org/apache/xmlrpc/main/xmlrpc-common.jar
(No such file or directory)
at
org.jboss.modules.xml.ModuleXmlParser.parseResourceRoot(ModuleXmlParser.java:891)
at
org.jboss.modules.xml.ModuleXmlParser.parseResources(ModuleXmlParser.java:735)
at
org.jboss.modules.xml.ModuleXmlParser.parseModuleContents(ModuleXmlParser.java:535)
at
org.jboss.modules.xml.ModuleXmlParser.parseDocument(ModuleXmlParser.java:340)
at
org.jboss.modules.xml.ModuleXmlParser.parseModuleXml(ModuleXmlParser.java:226)
... 22 more

2016-11-29 21:11:33,915-05 ERROR [org.jboss.msc.service.fail] (MSC service
thread 1-5) MSC01: Failed to start service
jboss.module.service."deployment.engine.ear.welcome.war".main:
org.jboss.msc.service.StartException in service
jboss.module.service."deployment.engine.ear.welcome.war".main: WFLYSRV0179:
Failed to load module: deployment.engine.ear.welcome.war:main
at
org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:91)
[wildfly-server-2.0.10.Final.jar:2.0.10.Final]
at
org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
[jboss-msc-1.2.6.Final.jar:1.2.6.Final]
at
org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
[jboss-msc-1.2.6.Final.jar:1.2.6.Final]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[rt.jar:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[rt.jar:1.8.0_101]
at

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

2016-11-29 Thread Yaniv Kaul

On Tue, Nov 29, 2016 at 6:33 PM, Sven Kieske  wrote:

> On 29/11/16 15:54, Martin Sivak wrote:
> > I actually checked the oVirt 4.0.0 and 4.0.5 release notes and I do
> > not see anything mentioning ovirt-cli or REST v3 deprecation. This
> > will (removal of RESTv3 support) affect the optimizer and quite
> > possibly even hosted engine setup.
>
> Hi,
>
> this seems to indicate that even core ovirt devs do not
> know about every feature which gets deprecated.
>
> Maybe this could be solved by sandro's suggestion
> to at least create an BZ for each deprecation.
>

Makes sense, even as a Tracker BZ, because there's the work to remove the
code, and it might be in several components, as well as Documentation.
Y.


> I would agree with that, but also like to propose
> an alternative way, because at the stage of bug creation
> the deprecation is already set in stone and can't be discussed further
> (I fear).
>
> So my proposal would be to mail to the devel list at least
> the proposal: "I/we want to get rid of feature/implementation X, use Z
> instead".
>
> This would give developers and users enough time to engage with the
> community, replacing dependencies or start a discussion if removal can't
> be done later.
>
> A clear deprecation guideline would also be helpful, I guess.
>
> Something like: Features will get marked as deprecated in release X.Y
> and removed in X.Y+2 (or whatever number you might chose).
>
> This would allow for much better preparation in the community and lead
> to less unpleasant surprises.
>
> HTH & keep up the good work!
>
> --
> Mit freundlichen Grüßen / Regards
>
> Sven Kieske
>
> Systemadministrator
> Mittwald CM Service GmbH & Co. KG
> Königsberger Straße 6
> 32339 Espelkamp
> T: +495772 293100
> F: +495772 29
> https://www.mittwald.de
> Geschäftsführer: Robert Meyer
> St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
> Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
>
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Nir Soffer

On Tue, Nov 29, 2016 at 5:09 PM, Arik Hadas  wrote:
>
>
> - Original Message -
>> On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas  wrote:
>> >
>> >
>> > - Original Message -
>> >> On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
>> >> >
>> >> >
>> >> > - Original Message -
>> >> >> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
>> >> >> > Hi All,
>> >> >> >
>> >> >> > We are working on something that is expected to have a big impact,
>> >> >> > hence
>> >> >> > this heads-up.
>> >> >> > First, we want you to be aware of this change and provide your
>> >> >> > feedback
>> >> >> > to
>> >> >> > make it as good as possible.
>> >> >> > Second, until the proposed mechanism is fully merged there will be a
>> >> >> > chase
>> >> >> > to cover all features unless new features are also implemented with
>> >> >> > the
>> >> >> > new mechanism. So please, if you are working on something that
>> >> >> > adds/changes something in the Libvirt's domain xml, do it with this
>> >> >> > new
>> >> >> > mechanism as well (first version would be merged soon).
>> >> >> >
>> >> >> > * Goal
>> >> >> > Creating Libvirt XML in the engine rather than in VDSM.
>> >> >> > ** Today's flow
>> >> >> > Engine: VM business entity -> VM properties map
>> >> >> > VDSM:   VM properties map  -> Libvirt XML
>> >> >> > ** Desired flow
>> >> >> > Engine: VM business entity -> Libvirt XML
>> >> >> >
>> >> >> > * Potential Benefits
>> >> >> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
>> >> >> > mistakes in the process.
>> >> >> > 2. Reduce the amount of code in VDSM.
>> >> >> > 3. Make VM related changes easier - today many of these changes need
>> >> >> > to
>> >> >> > be
>> >> >> > reviewed in 2 projects, this will eliminate the one that tends to
>> >> >> > take
>> >> >> > longer.
>> >> >> > 4. Prevent shortcuts in the form of VDSM-only changes that should be
>> >> >> > better
>> >> >> > reflected in the engine.
>> >> >> > 5. Not to re-generate the XML on each rerun attempt of VM
>> >> >> > run/migration.
>> >> >> > 6. Future - not to re-generate the XML on each attempt to auto-start
>> >> >> > HA
>> >> >> > VM
>> >> >> > when using vm-leases (need to make sure we're using the up-to-date VM
>> >> >> > configuration though).
>> >> >> > 7. We already found improvements and cleanups that could be made
>> >> >> > while
>> >> >> > touching this area (e.g., remove the boot order from devices in the
>> >> >> > database).
>> >> >> >
>> >> >> > * Challenges
>> >> >> > 1. Not to move host-specific information to the engine. For example,
>> >> >> > path
>> >> >> > to storage domain or sockets of channels.
>> >> >> >The solution is to use place-holders that will be replaced by
>> >> >> >VDSM.
>> >> >> > 2. Backward compatibility.
>> >> >> > 3. The more challenging part is the other direction - that will be
>> >> >> > the
>> >> >> > next
>> >> >> > phase.
>> >> >> >
>> >> >> > * Status
>> >> >> > As a first step, we began with producing the Libvirt XML in the
>> >> >> > engine
>> >> >> > by
>> >> >> > converting the VM properties map to XML in the engine [1]
>> >> >> > And using the XML that is received as an input in VDSM [2]
>> >> >> >
>> >> >> >
>> >> >> > [1] https://gerrit.ovirt.org/#/c/64473/
>> >> >> > [2] https://gerrit.ovirt.org/#/c/65182/
>> >> >>
>> >> >> I should start by saying that I love libvirt's domxml standard. Unlike
>> >> >> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
>> >> >> you are suggesting a positive step.
>> >> >>
>> >> >> However, Engine is much more complex than Vdsm. It is also our
>> >> >> single-point-of-failure, and where CPU is the most scarce. I am worried
>> >> >> that in the foreseeable future it would only make Engine bigger,
>> >> >> without
>> >> >> reducing the size and complexity of Vdsm.
>> >> >>
>> >> >> Before taking this move, we must map what Vdsm does, because that logic
>> >> >> would have to be copied into Engine. Few things pop up to mind:
>> >> >>
>> >> >> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>> >> >>   in XML format, or would it keep parsing them?
>> >> >
>> >> > Ideally, VDSM will report back the devices in XML format.
>> >> > The engine will then add the unmanaged devices and update the pci
>> >> > addresses.
>> >> > Need to put some more thoughts into this, though.
>> >> >
>> >> >>
>> >> >> - hot plug. Device xml should be generated by Engine, much like in the
>> >> >>   vm cteate flow
>> >> >
>> >> > Good point, I didn't think of hot plugs - right, they could be changed
>> >> > as
>> >> > well later on.
>> >> >
>> >> >>
>> >> >> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>> >> >>   that is connected no-where. Engine would need to care about what was
>> >> >>   up until now a vdsm-side implementation detail.
>> >> >
>> >> > Right, I almost finished to copy the creation of the network

[ovirt-devel] VDSM changes Linux memory dirty ratios - why?

2016-11-29 Thread Yaniv Kaul

It appears that VDSM changes the following params:
vm.dirty_ratio = 5
vm.dirty_background_ratio = 2

Any idea why? Because we use cache=none it's irrelevant anyway?
TIA,
Y.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] planned jenkins and resources downtime

2016-11-29 Thread Evgheni Dereveanchin

Maintenance completed, all services back up
and running. If you notice any issues please
inform me or open a ticket on jira.ovirt.org

Regards, 
Evgheni Dereveanchin 

- Original Message -
From: "Evgheni Dereveanchin" 
To: "infra" 
Cc: "devel" , us...@ovirt.org
Sent: Tuesday, 29 November, 2016 6:45:32 PM
Subject: planned jenkins and resources downtime

Hi everyone,

As part of a planned maintenance window I am going to 
apply OS updates and reboot the Jenkins master along 
with our resources file server.

Affected services:
 jenkins.ovirt.org
 resources.ovirt.org

This means that no new jobs will be started on the Jenkins
and our repositories will be unavailable for a short
period of time.

I will follow up on this email once all services 
are back up and running.

Regards, 
Evgheni Dereveanchin 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [JIRA] (OVIRT-416) [standard-ci] build-artifacts should reuse the artifacts built on check-merged if any

2016-11-29 Thread Vojtech Szocs



- Original Message -
> From: "Eyal Edri" 
> To: "Vojtech Szocs" 
> Cc: "Oved Ourfali" , "devel" 
> Sent: Sunday, November 27, 2016 11:28:08 AM
> Subject: Re: [JIRA] (OVIRT-416) [standard-ci] build-artifacts should reuse 
> the artifacts built on check-merged if any
> 
> Hi Vojtech,
> 
> I posted a patch [1] for this, so the change isn't merged yet.
> We thought of using the 'minimal' macro for it.
> 
> Can you send your suggestion to the patch?

Hi, sorry for late reply.

The patch looks good, my original idea was [all browsers x en_US],
but with "ovirt_build_minimal 1" it's actually [Firefox x en_US].

So +1 on that, but I see it's already merged =)

In automation/build-artifacts.sh, we can also remove the `BUILD_UT`
variable as it's not needed (taken care of by "ovirt_build_minimal").

> 
> 
> [1] https://gerrit.ovirt.org/#/c/67269/
> 
> On Fri, Nov 25, 2016 at 8:43 PM, Vojtech Szocs  wrote:
> 
> > Hi,
> >
> > > Any patches sent on this?
> >
> > I'd like to help.
> >
> > Looking at the recent Engine (master) build:
> >
> >   http://jenkins.ovirt.org/job/ovirt-engine_master_build-
> > artifacts-el7-x86_64/1691/
> >
> >   $ rpm -qlp ovirt-engine-webadmin-portal-4.1.0-0.0.master.
> > 20161125171318.gitae69c34.el7.centos.noarch.rpm
> >
> > it shows we still build all [browser x language] GWT permutations,
> > basically there are tons of .cache.js files in `webadmin.war`
> > directory (each one represents a single permutation).
> >
> > In automation/build-artifacts.sh, we could add some variable that
> > controls what kind of GWT build to do (e.g. GWT_BUILD_MODE):
> >
> > 1, snapshot GWT build: all supported browsers x en_US (only)
> >=> 3 x 1 = 3 GWT permutations
> >
> > 2, full GWT build: all supported browsers x all supported locales
> >=> 3 x 9 = 27 (!) GWT permutations
> >
> > With "snapshot GWT build", we should do:
> >
> >   rpmbuild \
> > .. existing stuff
> > -D "ovirt_build_locales 0" \
> > .. existing stuff
> >
> > GWT_BUILD_MODE can default to "full GWT build".
> >
> > But how do we override GWT_BUILD_MODE when doing non-release builds?
> >
> > Regards,
> > Vojtech
> >
> >
> > - Original Message -
> > > From: "eyal edri [Administrator] (oVirt JIRA)" <
> > j...@ovirt-jira.atlassian.net>
> > > To: in...@ovirt.org
> > > Sent: Sunday, November 6, 2016 4:29:02 PM
> > > Subject: [JIRA] (OVIRT-416) [standard-ci] build-artifacts should reuse
> > theartifacts built on check-merged if any
> > >
> > >
> > > [
> > > https://ovirt-jira.atlassian.net/browse/OVIRT-416?page=com.
> > atlassian.jira.plugin.system.issuetabpanels:comment-
> > tabpanel=22321#comment-22321
> > > ]
> > >
> > > eyal edri [Administrator] commented on OVIRT-416:
> > > -
> > >
> > > Any patches sent on this?
> > > Also, wanted to mention #OVIRT-751 is handling similar purpose with
> > enabling
> > > better maven caching, so it might suffice as a starting point in
> > optimizing
> > > ovirt-engine runtime for builds.
> > >
> > > > [standard-ci] build-artifacts should reuse the artifacts built on
> > > > check-merged if any
> > > > 
> > -
> > > >
> > > > Key: OVIRT-416
> > > > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-416
> > > > Project: oVirt - virtualization made easy
> > > >  Issue Type: Improvement
> > > >Reporter: David Caro Estevez
> > > >Assignee: infra
> > > >
> > > > Right now most projects just rebuild twice the artifacts, what for some
> > > > means a big waste of resources, if we could just reuse what was built
> > > > previously there would be no rebuilding.
> > > > Something to take into account is that the distributions that
> > check-merged
> > > > and build-artifacts run for might not match, so you can be running
> > > > check-merged only on fc23 but building artifacts for fc23, fc22 an el7
> > (as
> > > > an example), maybe we should not allow different distros on each
> > stage?|
> > >
> > >
> > >
> > > --
> > > This message was sent by Atlassian JIRA
> > > (v1000.499.4#100018)
> > > ___
> > > Infra mailing list
> > > in...@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/infra
> > >
> >
> 
> 
> 
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R
> Red Hat Israel
> 
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] planned jenkins and resources downtime

2016-11-29 Thread Evgheni Dereveanchin

Hi everyone,

As part of a planned maintenance window I am going to 
apply OS updates and reboot the Jenkins master along 
with our resources file server.

Affected services:
 jenkins.ovirt.org
 resources.ovirt.org

This means that no new jobs will be started on the Jenkins
and our repositories will be unavailable for a short
period of time.

I will follow up on this email once all services 
are back up and running.

Regards, 
Evgheni Dereveanchin 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

2016-11-29 Thread Sven Kieske

On 29/11/16 15:54, Martin Sivak wrote:
> I actually checked the oVirt 4.0.0 and 4.0.5 release notes and I do
> not see anything mentioning ovirt-cli or REST v3 deprecation. This
> will (removal of RESTv3 support) affect the optimizer and quite
> possibly even hosted engine setup.

Hi,

this seems to indicate that even core ovirt devs do not
know about every feature which gets deprecated.

Maybe this could be solved by sandro's suggestion
to at least create an BZ for each deprecation.

I would agree with that, but also like to propose
an alternative way, because at the stage of bug creation
the deprecation is already set in stone and can't be discussed further
(I fear).

So my proposal would be to mail to the devel list at least
the proposal: "I/we want to get rid of feature/implementation X, use Z
instead".

This would give developers and users enough time to engage with the
community, replacing dependencies or start a discussion if removal can't
be done later.

A clear deprecation guideline would also be helpful, I guess.

Something like: Features will get marked as deprecated in release X.Y
and removed in X.Y+2 (or whatever number you might chose).

This would allow for much better preparation in the community and lead
to less unpleasant surprises.

HTH & keep up the good work!

-- 
Mit freundlichen Grüßen / Regards

Sven Kieske

Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +495772 293100
F: +495772 29
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen

signature.asc
Description: OpenPGP digital signature
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] oVirt 4.1.0 beta1 compose planned

2016-11-29 Thread Sandro Bonazzola

Fyi oVirt developers,

An oVirt compose is planned for this Thursday 11:00 AM TLV time (10:00 AM
CET).
Please note this will be the first oVirt 4.1 beta compose and will be done
taking master snapshot in its state at compose time. If you've pending
fixes to be published please ping me at least 40 minutes before the
scheduled time.

A list of pending blockers is available here:
https://bugzilla.redhat.com/buglist.cgi?quicksearch=target_milestone%3A4.1.0%20flag%3Ablocker%20status%3Anew%2Cassigned%2Cpost

-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Allow editing hosted engine configuration on shared storage

2016-11-29 Thread Evgenia Tokar

Hi,

I'm working on a feature that will allow editing hosted engine
configuration on shared storage (currently it can't be easily accessed or
modified).

This is the link to the pull request for the feature page:
https://github.com/oVirt/ovirt-site/pull/621

Any comments are welcome.

Thanks,
Jenny Tokar
SLA / oVirt
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] Create soft-host-to-vm-affinity-support feature

2016-11-29 Thread Yanir Quinn

Hi,

I have posted a pull request for soft-host-to-vm-affinity-support feature :

https://github.com/oVirt/ovirt-site/pull/611

The feature comes to support the following use case:
*Certain set of VMs form a logical management unit should run on a certain
set of hosts for SLA or management (e.g. a separate rack for each
customer). The VMs can run anywhere in case the dedicated rack needs to be
turned off, but should return to their dedicated hosts once the rack is
back up.*

comments/remarks/other suggestions will be appreciated.

--
Yanir Quinn
SLA / ovirt
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Nir Soffer

On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas  wrote:
>
>
> - Original Message -
>> On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
>> >
>> >
>> > - Original Message -
>> >> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
>> >> > Hi All,
>> >> >
>> >> > We are working on something that is expected to have a big impact, hence
>> >> > this heads-up.
>> >> > First, we want you to be aware of this change and provide your feedback
>> >> > to
>> >> > make it as good as possible.
>> >> > Second, until the proposed mechanism is fully merged there will be a
>> >> > chase
>> >> > to cover all features unless new features are also implemented with the
>> >> > new mechanism. So please, if you are working on something that
>> >> > adds/changes something in the Libvirt's domain xml, do it with this new
>> >> > mechanism as well (first version would be merged soon).
>> >> >
>> >> > * Goal
>> >> > Creating Libvirt XML in the engine rather than in VDSM.
>> >> > ** Today's flow
>> >> > Engine: VM business entity -> VM properties map
>> >> > VDSM:   VM properties map  -> Libvirt XML
>> >> > ** Desired flow
>> >> > Engine: VM business entity -> Libvirt XML
>> >> >
>> >> > * Potential Benefits
>> >> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
>> >> > mistakes in the process.
>> >> > 2. Reduce the amount of code in VDSM.
>> >> > 3. Make VM related changes easier - today many of these changes need to
>> >> > be
>> >> > reviewed in 2 projects, this will eliminate the one that tends to take
>> >> > longer.
>> >> > 4. Prevent shortcuts in the form of VDSM-only changes that should be
>> >> > better
>> >> > reflected in the engine.
>> >> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
>> >> > 6. Future - not to re-generate the XML on each attempt to auto-start HA
>> >> > VM
>> >> > when using vm-leases (need to make sure we're using the up-to-date VM
>> >> > configuration though).
>> >> > 7. We already found improvements and cleanups that could be made while
>> >> > touching this area (e.g., remove the boot order from devices in the
>> >> > database).
>> >> >
>> >> > * Challenges
>> >> > 1. Not to move host-specific information to the engine. For example,
>> >> > path
>> >> > to storage domain or sockets of channels.
>> >> >The solution is to use place-holders that will be replaced by VDSM.
>> >> > 2. Backward compatibility.
>> >> > 3. The more challenging part is the other direction - that will be the
>> >> > next
>> >> > phase.
>> >> >
>> >> > * Status
>> >> > As a first step, we began with producing the Libvirt XML in the engine
>> >> > by
>> >> > converting the VM properties map to XML in the engine [1]
>> >> > And using the XML that is received as an input in VDSM [2]
>> >> >
>> >> >
>> >> > [1] https://gerrit.ovirt.org/#/c/64473/
>> >> > [2] https://gerrit.ovirt.org/#/c/65182/
>> >>
>> >> I should start by saying that I love libvirt's domxml standard. Unlike
>> >> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
>> >> you are suggesting a positive step.
>> >>
>> >> However, Engine is much more complex than Vdsm. It is also our
>> >> single-point-of-failure, and where CPU is the most scarce. I am worried
>> >> that in the foreseeable future it would only make Engine bigger, without
>> >> reducing the size and complexity of Vdsm.
>> >>
>> >> Before taking this move, we must map what Vdsm does, because that logic
>> >> would have to be copied into Engine. Few things pop up to mind:
>> >>
>> >> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>> >>   in XML format, or would it keep parsing them?
>> >
>> > Ideally, VDSM will report back the devices in XML format.
>> > The engine will then add the unmanaged devices and update the pci
>> > addresses.
>> > Need to put some more thoughts into this, though.
>> >
>> >>
>> >> - hot plug. Device xml should be generated by Engine, much like in the
>> >>   vm cteate flow
>> >
>> > Good point, I didn't think of hot plugs - right, they could be changed as
>> > well later on.
>> >
>> >>
>> >> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>> >>   that is connected no-where. Engine would need to care about what was
>> >>   up until now a vdsm-side implementation detail.
>> >
>> > Right, I almost finished to copy the creation of the network interfaces to
>> > the engine.
>> > This knowledge that you refer to will only be in the module that creates
>> > the XML, it doesn't seem to be much of an issue.
>> >
>> >>
>> >> - storage path. this was mentioned above, but actually, the paths are
>> >>   the same on all hosts. We inteded to have an abstraction layer there,
>> >>   but we never ever used it. All volumes sit under
>> >>   /rhev/data-center/poolID/domainID/imageID/volumeID
>> >>   Basically, Engine can hard-code this in the domxml, and no one would
>> >>   notice.
>>
>> This is wrong, and engine cannot

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

2016-11-29 Thread Martin Sivak

I actually checked the oVirt 4.0.0 and 4.0.5 release notes and I do
not see anything mentioning ovirt-cli or REST v3 deprecation. This
will (removal of RESTv3 support) affect the optimizer and quite
possibly even hosted engine setup.

Martin

On Tue, Nov 29, 2016 at 3:26 PM, Sandro Bonazzola  wrote:
> Moving to devel since this needs developer attention
>
> On Tue, Nov 29, 2016 at 2:50 PM, Logan Kuhn  wrote:
>>
>> Is this something that was officially announced and I've missed?  This is
>> the first time I'm hearing about the removal of the cli
>>
>
> I tend to agree with Logan here, we have a lack of communication on what's
> going on.
> We haven't weekly status meetings anymore, the experiment on office hours
> didn't work and progress reports to mailing lists are not sent anymore.
> We need to rethink the way we propagate information.
>
> Up to now the only way we have to give info on coming deprecation is the
> release notes page which is built from bugzilla doctext so if no other way
> is used to communicate changes, at least please use a bug.
>
>
>>
>> Regards,
>> Logan
>>
>
> --
> Sandro Bonazzola
> Better technology. Faster innovation. Powered by community collaboration.
> See how it works at redhat.com
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Arik Hadas



- Original Message -
> On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
> >
> >
> > - Original Message -
> >> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> >> > Hi All,
> >> >
> >> > We are working on something that is expected to have a big impact, hence
> >> > this heads-up.
> >> > First, we want you to be aware of this change and provide your feedback
> >> > to
> >> > make it as good as possible.
> >> > Second, until the proposed mechanism is fully merged there will be a
> >> > chase
> >> > to cover all features unless new features are also implemented with the
> >> > new mechanism. So please, if you are working on something that
> >> > adds/changes something in the Libvirt's domain xml, do it with this new
> >> > mechanism as well (first version would be merged soon).
> >> >
> >> > * Goal
> >> > Creating Libvirt XML in the engine rather than in VDSM.
> >> > ** Today's flow
> >> > Engine: VM business entity -> VM properties map
> >> > VDSM:   VM properties map  -> Libvirt XML
> >> > ** Desired flow
> >> > Engine: VM business entity -> Libvirt XML
> >> >
> >> > * Potential Benefits
> >> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >> > mistakes in the process.
> >> > 2. Reduce the amount of code in VDSM.
> >> > 3. Make VM related changes easier - today many of these changes need to
> >> > be
> >> > reviewed in 2 projects, this will eliminate the one that tends to take
> >> > longer.
> >> > 4. Prevent shortcuts in the form of VDSM-only changes that should be
> >> > better
> >> > reflected in the engine.
> >> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> >> > 6. Future - not to re-generate the XML on each attempt to auto-start HA
> >> > VM
> >> > when using vm-leases (need to make sure we're using the up-to-date VM
> >> > configuration though).
> >> > 7. We already found improvements and cleanups that could be made while
> >> > touching this area (e.g., remove the boot order from devices in the
> >> > database).
> >> >
> >> > * Challenges
> >> > 1. Not to move host-specific information to the engine. For example,
> >> > path
> >> > to storage domain or sockets of channels.
> >> >The solution is to use place-holders that will be replaced by VDSM.
> >> > 2. Backward compatibility.
> >> > 3. The more challenging part is the other direction - that will be the
> >> > next
> >> > phase.
> >> >
> >> > * Status
> >> > As a first step, we began with producing the Libvirt XML in the engine
> >> > by
> >> > converting the VM properties map to XML in the engine [1]
> >> > And using the XML that is received as an input in VDSM [2]
> >> >
> >> >
> >> > [1] https://gerrit.ovirt.org/#/c/64473/
> >> > [2] https://gerrit.ovirt.org/#/c/65182/
> >>
> >> I should start by saying that I love libvirt's domxml standard. Unlike
> >> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
> >> you are suggesting a positive step.
> >>
> >> However, Engine is much more complex than Vdsm. It is also our
> >> single-point-of-failure, and where CPU is the most scarce. I am worried
> >> that in the foreseeable future it would only make Engine bigger, without
> >> reducing the size and complexity of Vdsm.
> >>
> >> Before taking this move, we must map what Vdsm does, because that logic
> >> would have to be copied into Engine. Few things pop up to mind:
> >>
> >> - pci addresses. would Vdsm report back the libvirt-assigned addresses
> >>   in XML format, or would it keep parsing them?
> >
> > Ideally, VDSM will report back the devices in XML format.
> > The engine will then add the unmanaged devices and update the pci
> > addresses.
> > Need to put some more thoughts into this, though.
> >
> >>
> >> - hot plug. Device xml should be generated by Engine, much like in the
> >>   vm cteate flow
> >
> > Good point, I didn't think of hot plugs - right, they could be changed as
> > well later on.
> >
> >>
> >> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
> >>   that is connected no-where. Engine would need to care about what was
> >>   up until now a vdsm-side implementation detail.
> >
> > Right, I almost finished to copy the creation of the network interfaces to
> > the engine.
> > This knowledge that you refer to will only be in the module that creates
> > the XML, it doesn't seem to be much of an issue.
> >
> >>
> >> - storage path. this was mentioned above, but actually, the paths are
> >>   the same on all hosts. We inteded to have an abstraction layer there,
> >>   but we never ever used it. All volumes sit under
> >>   /rhev/data-center/poolID/domainID/imageID/volumeID
> >>   Basically, Engine can hard-code this in the domxml, and no one would
> >>   notice.
> 
> This is wrong, and engine cannot hard code this or anything else.
> 
> Engine should can describe only what is knows about disks, only vdsm
> can add the disk xml.

Of course, the engine will describe only the information

[ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

2016-11-29 Thread Sandro Bonazzola

Moving to devel since this needs developer attention

On Tue, Nov 29, 2016 at 2:50 PM, Logan Kuhn  wrote:

> Is this something that was officially announced and I've missed?  This is
> the first time I'm hearing about the removal of the cli
>
>
I tend to agree with Logan here, we have a lack of communication on what's
going on.
We haven't weekly status meetings anymore, the experiment on office hours
didn't work and progress reports to mailing lists are not sent anymore.
We need to rethink the way we propagate information.

Up to now the only way we have to give info on coming deprecation is the
release notes page which is built from bugzilla doctext so if no other way
is used to communicate changes, at least please use a bug.



> Regards,
> Logan
>
>
-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

2016-11-29 Thread Nir Soffer

On Tue, Nov 29, 2016 at 3:26 PM, Ramesh Nachimuthu  wrote:
> Hi,
>
>  I am trying to run a python script '/sbin/gluster-eventsapi' in vdsm verb 
> which internally imports some python modules from 
> /usr/lib/python2.7/site-packages/gluster/cliutils/. But it fails with the 
> import error. Following error is seen in the supervdsm log.
>
>
> MainProcess|Thread-1::DEBUG::2016-11-28 
> 16:54:35,130::commands::93::root::(execCmd) FAILED:  = 'Traceback (most 
> recent call last):\n  File "/sbin/gluster-eventsapi", line 25, in \n  
>   from gluster.cliutils.cliutils import (Cmd, execute, node_output_ok, 
> node_output_notok,\nImportError: No module named cliutils.cliutils\n';  = 
> 1
>
>
> I think the import statement "from gluster.cliutils.cliutils import (Cmd, 
> execute, node_output_ok, node_output_notok)" in the python script resolves to 
> '/usr/share/vdsm/gluster' instead of 
> /usr/lib/python2.7/site-packages/gluster/cliutils.
>
> I see the following in python sys.path while executing a python script from 
> vdsm.
>
> ['/usr/libexec/glusterfs', '/usr/share/vdsm', '/usr/lib64/python27.zip', 
> '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', 
> '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', 
> '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', 
> '/usr/lib64/python2.7/site-packages/gtk-2.0', 
> '/usr/lib/python2.7/site-packages']
>
> Looks like '/usr/share/vdsm' take precedence over 
> '/usr/lib64/python2.7/site-packages'.
>
> Can someone suggests a way to fix this issue?
>
> Note: '/sbin/gluster-eventsapi' works perfectly while running directly from 
> CLI.
>
> Related vdsm patch: 
> https://gerrit.ovirt.org/#/c/67168/2/vdsm/gluster/events.py

If an import fails because /usr/share/vdsm/foo hides a package in
/usr/lib/python2.7/site-packages/vdsm it means that you have a wrong
import, you need to import anything like this:

from vdsm.foo import bar

So names in /usr/share/vdsm/* cannot hide names in vdsm package.

Nir
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] performance improvements and gwt-rpc switch

2016-11-29 Thread Greg Sheremeta

On Tue, Nov 29, 2016 at 4:40 AM, Yaniv Kaul  wrote:

>
>  is a graph that shows gwt-rpc performing slightly better than de-rpc.
>> Memory consumption is about the same -- gwt-rpc is just a faster rpc
>> implementation.
>>
>
> I'm wondering if we have any data about de-rpc vs. gwt-rpc under WAN
> conditions. With latency (say, 70ms, based on [1]) and possibly some packet
> loss (0.5% should suffice).
> Y.
>
> [1] http://www.internettrafficreport.com/30day.htm
>
>
We did WAN testing for the memory leaks, but not for the rpc change. (The
ask got misinterpreted by me -- my bad.)

I'll get you some results for this this week.

Greg
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

2016-11-29 Thread Ramesh Nachimuthu

Hi, 

 I am trying to run a python script '/sbin/gluster-eventsapi' in vdsm verb 
which internally imports some python modules from 
/usr/lib/python2.7/site-packages/gluster/cliutils/. But it fails with the 
import error. Following error is seen in the supervdsm log.


MainProcess|Thread-1::DEBUG::2016-11-28 
16:54:35,130::commands::93::root::(execCmd) FAILED:  = 'Traceback (most 
recent call last):\n  File "/sbin/gluster-eventsapi", line 25, in \n
from gluster.cliutils.cliutils import (Cmd, execute, node_output_ok, 
node_output_notok,\nImportError: No module named cliutils.cliutils\n';  = 1


I think the import statement "from gluster.cliutils.cliutils import (Cmd, 
execute, node_output_ok, node_output_notok)" in the python script resolves to 
'/usr/share/vdsm/gluster' instead of 
/usr/lib/python2.7/site-packages/gluster/cliutils. 

I see the following in python sys.path while executing a python script from 
vdsm.

['/usr/libexec/glusterfs', '/usr/share/vdsm', '/usr/lib64/python27.zip', 
'/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', 
'/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', 
'/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', 
'/usr/lib64/python2.7/site-packages/gtk-2.0', 
'/usr/lib/python2.7/site-packages']

Looks like '/usr/share/vdsm' take precedence over 
'/usr/lib64/python2.7/site-packages'. 

Can someone suggests a way to fix this issue?

Note: '/sbin/gluster-eventsapi' works perfectly while running directly from CLI.

Related vdsm patch: https://gerrit.ovirt.org/#/c/67168/2/vdsm/gluster/events.py


Regards,
Ramesh
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [Call for Vote] moVirt as a Full oVirt Project

2016-11-29 Thread Eldad Marciano

+1

On Tue, Nov 29, 2016 at 2:23 PM, Sandro Bonazzola 
wrote:

> Brian, Board, I think we have enough +1.
>
> On Tue, Nov 22, 2016 at 10:02 AM, Jakub Niedermertl 
> wrote:
>
>> +1
>>
>> - Original Message -
>> > From: "Roman Mohr" 
>> > To: "Brian Proffitt" 
>> > Cc: "Michal Skrivanek" , bo...@ovirt.org, "devel"
>> 
>> > Sent: Tuesday, November 22, 2016 9:30:23 AM
>> > Subject: Re: [ovirt-devel] [Call for Vote] moVirt as a Full oVirt
>> Project
>> >
>> > +1
>> >
>> > On Mon, Nov 21, 2016 at 6:07 PM, Brian Proffitt < bprof...@redhat.com >
>> > wrote:
>> >
>> >
>> >
>> > All:
>> >
>> > The moVirt Project was initially accepted as an oVirt incubator project
>> in
>> > February 2015. It has been a successful subproject for quite some time
>> and
>> > it is well due for being accepted as a full oVirt project. I believe it
>> is
>> > appropriate to post a Call for Vote on the Devel and Board lists.
>> >
>> > http://www.ovirt.org/develop/projects/project-movirt/
>> >
>> > A “healthy” project, as determined by the oVirt Board, can be found at
>> > http://www.ovirt.org/develop/projects/adding-a-new-project/
>> >
>> > Voting will be open until 1200 UTC Nov. 30, 2016. A net total of +7
>> votes
>> > should be received to formalize this project as an full oVirt project.
>> > Please use the following vote process:
>> >
>> > +1
>> > Yes, agree, or the action should be performed. On some issues, this
>> vote must
>> > only be given after the voter has tested the action on their own
>> system(s).
>> >
>> > ±0
>> > Abstain, no opinion, or I am happy to let the other group members
>> decide this
>> > issue. An abstention may have detrimental affects if too many people
>> > abstain.
>> >
>> > -1
>> > No, I veto this action. All vetos must include an explanation of why
>> the veto
>> > is appropriate. A veto with no explanation is void.
>> >
>> > Thank you!
>> >
>> > Brian Proffitt
>> > Principal Community Analyst
>> > Open Source and Standards
>> > @TheTechScribe
>> > 574.383.9BKP
>> >
>> > ___
>> > Devel mailing list
>> > Devel@ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>> >
>> >
>> > ___
>> > Devel mailing list
>> > Devel@ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>> ___
>> Devel mailing list
>> Devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> Sandro Bonazzola
> Better technology. Faster innovation. Powered by community collaboration.
> See how it works at redhat.com
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>



-- 
-Eldad
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Nir Soffer

On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas  wrote:
>
>
> - Original Message -
>> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
>> > Hi All,
>> >
>> > We are working on something that is expected to have a big impact, hence
>> > this heads-up.
>> > First, we want you to be aware of this change and provide your feedback to
>> > make it as good as possible.
>> > Second, until the proposed mechanism is fully merged there will be a chase
>> > to cover all features unless new features are also implemented with the
>> > new mechanism. So please, if you are working on something that
>> > adds/changes something in the Libvirt's domain xml, do it with this new
>> > mechanism as well (first version would be merged soon).
>> >
>> > * Goal
>> > Creating Libvirt XML in the engine rather than in VDSM.
>> > ** Today's flow
>> > Engine: VM business entity -> VM properties map
>> > VDSM:   VM properties map  -> Libvirt XML
>> > ** Desired flow
>> > Engine: VM business entity -> Libvirt XML
>> >
>> > * Potential Benefits
>> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
>> > mistakes in the process.
>> > 2. Reduce the amount of code in VDSM.
>> > 3. Make VM related changes easier - today many of these changes need to be
>> > reviewed in 2 projects, this will eliminate the one that tends to take
>> > longer.
>> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
>> > reflected in the engine.
>> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
>> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
>> > when using vm-leases (need to make sure we're using the up-to-date VM
>> > configuration though).
>> > 7. We already found improvements and cleanups that could be made while
>> > touching this area (e.g., remove the boot order from devices in the
>> > database).
>> >
>> > * Challenges
>> > 1. Not to move host-specific information to the engine. For example, path
>> > to storage domain or sockets of channels.
>> >The solution is to use place-holders that will be replaced by VDSM.
>> > 2. Backward compatibility.
>> > 3. The more challenging part is the other direction - that will be the next
>> > phase.
>> >
>> > * Status
>> > As a first step, we began with producing the Libvirt XML in the engine by
>> > converting the VM properties map to XML in the engine [1]
>> > And using the XML that is received as an input in VDSM [2]
>> >
>> >
>> > [1] https://gerrit.ovirt.org/#/c/64473/
>> > [2] https://gerrit.ovirt.org/#/c/65182/
>>
>> I should start by saying that I love libvirt's domxml standard. Unlike
>> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
>> you are suggesting a positive step.
>>
>> However, Engine is much more complex than Vdsm. It is also our
>> single-point-of-failure, and where CPU is the most scarce. I am worried
>> that in the foreseeable future it would only make Engine bigger, without
>> reducing the size and complexity of Vdsm.
>>
>> Before taking this move, we must map what Vdsm does, because that logic
>> would have to be copied into Engine. Few things pop up to mind:
>>
>> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>>   in XML format, or would it keep parsing them?
>
> Ideally, VDSM will report back the devices in XML format.
> The engine will then add the unmanaged devices and update the pci addresses.
> Need to put some more thoughts into this, though.
>
>>
>> - hot plug. Device xml should be generated by Engine, much like in the
>>   vm cteate flow
>
> Good point, I didn't think of hot plugs - right, they could be changed as 
> well later on.
>
>>
>> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>>   that is connected no-where. Engine would need to care about what was
>>   up until now a vdsm-side implementation detail.
>
> Right, I almost finished to copy the creation of the network interfaces to 
> the engine.
> This knowledge that you refer to will only be in the module that creates the 
> XML, it doesn't seem to be much of an issue.
>
>>
>> - storage path. this was mentioned above, but actually, the paths are
>>   the same on all hosts. We inteded to have an abstraction layer there,
>>   but we never ever used it. All volumes sit under
>>   /rhev/data-center/poolID/domainID/imageID/volumeID
>>   Basically, Engine can hard-code this in the domxml, and no one would
>>   notice.

This is wrong, and engine cannot hard code this or anything else.

Engine should can describe only what is knows about disks, only vdsm
can add the disk xml.

>
> But I see that LUN and cinder disks are represented differently (not as PDIV) 
> - I'll check this.

Of course, and even disks using DIV can modified in by vdsm, for
example glusterfs
using libgfapi.

>
>>
>> - OvS. Recently, we have changed how VMs can be connected to their
>>   network. It is possible (albeit not recommended yet!) to connect a VM
>>

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Arik Hadas



- Original Message -
> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> > Hi All,
> > 
> > We are working on something that is expected to have a big impact, hence
> > this heads-up.
> > First, we want you to be aware of this change and provide your feedback to
> > make it as good as possible.
> > Second, until the proposed mechanism is fully merged there will be a chase
> > to cover all features unless new features are also implemented with the
> > new mechanism. So please, if you are working on something that
> > adds/changes something in the Libvirt's domain xml, do it with this new
> > mechanism as well (first version would be merged soon).
> > 
> > * Goal
> > Creating Libvirt XML in the engine rather than in VDSM.
> > ** Today's flow
> > Engine: VM business entity -> VM properties map
> > VDSM:   VM properties map  -> Libvirt XML
> > ** Desired flow
> > Engine: VM business entity -> Libvirt XML
> > 
> > * Potential Benefits
> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> > mistakes in the process.
> > 2. Reduce the amount of code in VDSM.
> > 3. Make VM related changes easier - today many of these changes need to be
> > reviewed in 2 projects, this will eliminate the one that tends to take
> > longer.
> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
> > reflected in the engine.
> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
> > when using vm-leases (need to make sure we're using the up-to-date VM
> > configuration though).
> > 7. We already found improvements and cleanups that could be made while
> > touching this area (e.g., remove the boot order from devices in the
> > database).
> > 
> > * Challenges
> > 1. Not to move host-specific information to the engine. For example, path
> > to storage domain or sockets of channels.
> >The solution is to use place-holders that will be replaced by VDSM.
> > 2. Backward compatibility.
> > 3. The more challenging part is the other direction - that will be the next
> > phase.
> > 
> > * Status
> > As a first step, we began with producing the Libvirt XML in the engine by
> > converting the VM properties map to XML in the engine [1]
> > And using the XML that is received as an input in VDSM [2]
> > 
> > 
> > [1] https://gerrit.ovirt.org/#/c/64473/
> > [2] https://gerrit.ovirt.org/#/c/65182/
> 
> I should start by saying that I love libvirt's domxml standard. Unlike
> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
> you are suggesting a positive step.
> 
> However, Engine is much more complex than Vdsm. It is also our
> single-point-of-failure, and where CPU is the most scarce. I am worried
> that in the foreseeable future it would only make Engine bigger, without
> reducing the size and complexity of Vdsm.
> 
> Before taking this move, we must map what Vdsm does, because that logic
> would have to be copied into Engine. Few things pop up to mind:
> 
> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>   in XML format, or would it keep parsing them?

Ideally, VDSM will report back the devices in XML format.
The engine will then add the unmanaged devices and update the pci addresses.
Need to put some more thoughts into this, though.

> 
> - hot plug. Device xml should be generated by Engine, much like in the
>   vm cteate flow

Good point, I didn't think of hot plugs - right, they could be changed as well 
later on.

> 
> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>   that is connected no-where. Engine would need to care about what was
>   up until now a vdsm-side implementation detail.

Right, I almost finished to copy the creation of the network interfaces to the 
engine.
This knowledge that you refer to will only be in the module that creates the 
XML, it doesn't seem to be much of an issue.

> 
> - storage path. this was mentioned above, but actually, the paths are
>   the same on all hosts. We inteded to have an abstraction layer there,
>   but we never ever used it. All volumes sit under
>   /rhev/data-center/poolID/domainID/imageID/volumeID
>   Basically, Engine can hard-code this in the domxml, and no one would
>   notice.

But I see that LUN and cinder disks are represented differently (not as PDIV) - 
I'll check this.

> 
> - OvS. Recently, we have changed how VMs can be connected to their
>   network. It is possible (albeit not recommended yet!) to connect a VM
>   to an OvS instead of Linux bridges. This is done without Engine really
>   caring, or knowing how the domxml is modified.

Yeah, I saw that. The only complication I see at this point is that for OvS we 
create more elements than only the 'source' element.
I believe that we could use a place-holder that contains the network name and 
replace it with the tags that are needed for SR-IOV, OvS and ordinary 
interfaces, no?
This seems to

Re: [ovirt-devel] ovirt-engine-api-explorer - webpack, command not found

2016-11-29 Thread Juan Hernández

On 11/29/2016 11:52 AM, Sandro Bonazzola wrote:
> The following job is failing on missing webpack:
> http://jenkins.ovirt.org/job/ovirt-engine-api-explorer_master_build-artifacts-el7-x86_64/10/console
> 
> can you please fix?
> 

This patch should address the issue:

  Use Node.js setup-env.sh
  https://gerrit.ovirt.org/67500

-- 
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] ovirt-engine-api-explorer - webpack, command not found

2016-11-29 Thread Sandro Bonazzola

The following job is failing on missing webpack:
http://jenkins.ovirt.org/job/ovirt-engine-api-explorer_master_build-artifacts-el7-x86_64/10/console

can you please fix?

-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

2016-11-29 Thread Dan Kenigsberg

On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> Hi All,
> 
> We are working on something that is expected to have a big impact, hence this 
> heads-up.
> First, we want you to be aware of this change and provide your feedback to 
> make it as good as possible.
> Second, until the proposed mechanism is fully merged there will be a chase to 
> cover all features unless new features are also implemented with the new 
> mechanism. So please, if you are working on something that adds/changes 
> something in the Libvirt's domain xml, do it with this new mechanism as well 
> (first version would be merged soon).
> 
> * Goal
> Creating Libvirt XML in the engine rather than in VDSM.
> ** Today's flow
> Engine: VM business entity -> VM properties map
> VDSM:   VM properties map  -> Libvirt XML
> ** Desired flow
> Engine: VM business entity -> Libvirt XML
> 
> * Potential Benefits
> 1. Reduce the number of conversions from 2 to 1, reducing chances for 
> mistakes in the process.
> 2. Reduce the amount of code in VDSM.
> 3. Make VM related changes easier - today many of these changes need to be 
> reviewed in 2 projects, this will eliminate the one that tends to take longer.
> 4. Prevent shortcuts in the form of VDSM-only changes that should be better 
> reflected in the engine.
> 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> 6. Future - not to re-generate the XML on each attempt to auto-start HA VM 
> when using vm-leases (need to make sure we're using the up-to-date VM 
> configuration though).
> 7. We already found improvements and cleanups that could be made while 
> touching this area (e.g., remove the boot order from devices in the database).
> 
> * Challenges
> 1. Not to move host-specific information to the engine. For example, path to 
> storage domain or sockets of channels.
>The solution is to use place-holders that will be replaced by VDSM.
> 2. Backward compatibility.
> 3. The more challenging part is the other direction - that will be the next 
> phase.
> 
> * Status
> As a first step, we began with producing the Libvirt XML in the engine by 
> converting the VM properties map to XML in the engine [1]
> And using the XML that is received as an input in VDSM [2]
> 
> 
> [1] https://gerrit.ovirt.org/#/c/64473/
> [2] https://gerrit.ovirt.org/#/c/65182/

I should start by saying that I love libvirt's domxml standard. Unlike
Vdsm's API, it is a real *standard* for defining VMs. In this regards,
you are suggesting a positive step.

However, Engine is much more complex than Vdsm. It is also our
single-point-of-failure, and where CPU is the most scarce. I am worried
that in the foreseeable future it would only make Engine bigger, without
reducing the size and complexity of Vdsm.

Before taking this move, we must map what Vdsm does, because that logic
would have to be copied into Engine. Few things pop up to mind:

- pci addresses. would Vdsm report back the libvirt-assigned addresses
  in XML format, or would it keep parsing them?

- hot plug. Device xml should be generated by Engine, much like in the
  vm cteate flow

- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
  that is connected no-where. Engine would need to care about what was
  up until now a vdsm-side implementation detail.

- storage path. this was mentioned above, but actually, the paths are
  the same on all hosts. We inteded to have an abstraction layer there,
  but we never ever used it. All volumes sit under
  /rhev/data-center/poolID/domainID/imageID/volumeID
  Basically, Engine can hard-code this in the domxml, and no one would
  notice.

- OvS. Recently, we have changed how VMs can be connected to their
  network. It is possible (albeit not recommended yet!) to connect a VM
  to an OvS instead of Linux bridges. This is done without Engine really
  caring, or knowing how the domxml is modified.

- minor tweaks. exposing a new feature into Engine's UI is hard. Over
  the years, few tweaks have been pushed as custom properties.
  there are not many (I see now only sndbuf, queues, viodiskcache,
  vhost) but the implementation should make sure they are not forgotten.

Maybe, Vdsm should consider Engine's domxml only as a "recomendation"
and modify it based on its hooks and custom properties. This can
surprise Engine, a defies the pupose of having xml-building logic moved
away from Vdsm.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] VDSM changes Linux memory dirty ratios - why?

Re: [ovirt-devel] master won't start

Re: [ovirt-devel] master won't start

Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

Re: [ovirt-devel] master won't start

[ovirt-devel] master won't start

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[ovirt-devel] VDSM changes Linux memory dirty ratios - why?

Re: [ovirt-devel] planned jenkins and resources downtime

Re: [ovirt-devel] [JIRA] (OVIRT-416) [standard-ci] build-artifacts should reuse the artifacts built on check-merged if any

[ovirt-devel] planned jenkins and resources downtime

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

[ovirt-devel] oVirt 4.1.0 beta1 compose planned

[ovirt-devel] Allow editing hosted engine configuration on shared storage

[ovirt-devel] Create soft-host-to-vm-affinity-support feature

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

Re: [ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[ovirt-devel] [RFC] Communicating within the community [was Re: [ovirt-users] I wrote an oVirt thing]

Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

Re: [ovirt-devel] performance improvements and gwt-rpc switch

[ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

Re: [ovirt-devel] [Call for Vote] moVirt as a Full oVirt Project

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

Re: [ovirt-devel] ovirt-engine-api-explorer - webpack, command not found

[ovirt-devel] ovirt-engine-api-explorer - webpack, command not found

Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

29 matches

Site Navigation

Mail list logo

Footer information