Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Hi, sorry for digging up an old tread, but I have a problem with proposed way to preserve vdsm.conf between host redeployments I've created a file /etc/ovirt-host-deploy.conf.d/migration-bw.conf on ovirt-engine with contents: [environment:enforce] VDSM_CONFIG/vars/migration_max_bandwidth=str:300 Then i've tried to add a new host via GUI - it was added, but with default vdsm.conf, i.e. without my custom option migration_max_bandwidth Also tried to restart ovirt-engine and reinstalling host - same result, no custom options, only default vdsm.conf content. Am i missing something? Freshly installed ovirt 3.5-pre, centos 6.5 as engine, centos7 as hosts Yuriy Demchenko On 08/05/2014 10:45 PM, Alon Bar-Lev wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: ybronhei ybron...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 9:36:24 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos and should stop here.. , and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. you can create a file /etc/ovirt-host-deploy.conf.d/40-xxx.conf --- VDSM_CONFIG/section/key=str:content --- this will create a proper vdsm.conf when host-deploy is initiated. you should now use the rest api to initiate host-deploy. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). right, but you should let this process install packages and manage configuration. What I've noticed is that after step #3, after a host is added by ovirt-engine, the vdsm.conf file is reset to default and I have to reapply Puppet before it can be used as the one of my Data Storage Domains requires iSER (not available over TCP). right, see above. What would be the workflow using ovirt-host-deploy? Thus far I've had to piece together my workflow based on the documentation and filling in blanks where possible since I do require customizations to vdsm.conf and the documented workflow of adding a host via web UI does not allow for such customization. Thanks, - Trey [1] - https://github.com/treydock/puppet-ovirt (README not fully updated as still working out how to use Puppet with oVirt) On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
afaik el6 and openssh support pkcs#8 for ages. at least I have no issue using pkcs#8 formatted keys with el6 hosts. Am 21.08.2014 22:05, schrieb Alon Bar-Lev: I hope el6 will have newer openssh with support for the PKCS8 format. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Sven Kieske s.kie...@mittwald.de To: users@ovirt.org Sent: Friday, August 22, 2014 10:32:05 AM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options afaik el6 and openssh support pkcs#8 for ages. at least I have no issue using pkcs#8 formatted keys with el6 hosts. the ssh-keygen does not. Am 21.08.2014 22:05, schrieb Alon Bar-Lev: I hope el6 will have newer openssh with support for the PKCS8 format. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
well yeah, it does not generate pkcs#8 by default but you can easily convert existing keys via openssl: openssl pkcs8 -topk8 -v2 des3 \ -in test_rsa_key.old -passin 'pass:super secret passphrase' \ -out test_rsa_key -passout 'pass:super secret passphrase' see this page for more details: http://martin.kleppmann.com/2013/05/24/improving-security-of-ssh-private-keys.html newer ssh-keygen versions use PBKDF2 by default and not MD5 anymore. HTH Am 22.08.2014 10:51, schrieb Alon Bar-Lev: the ssh-keygen does not. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
you are hijacking this thread... but anyway... please refer to the original question, how to easily convert X.509 certificate to SSH public key. the best method should avoid using the private key. newer ssh-keygen supports exactly that. - Original Message - From: Sven Kieske s.kie...@mittwald.de To: Alon Bar-Lev alo...@redhat.com Cc: users@ovirt.org Sent: Friday, August 22, 2014 1:24:17 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options well yeah, it does not generate pkcs#8 by default but you can easily convert existing keys via openssl: openssl pkcs8 -topk8 -v2 des3 \ -in test_rsa_key.old -passin 'pass:super secret passphrase' \ -out test_rsa_key -passout 'pass:super secret passphrase' see this page for more details: http://martin.kleppmann.com/2013/05/24/improving-security-of-ssh-private-keys.html newer ssh-keygen versions use PBKDF2 by default and not MD5 anymore. HTH Am 22.08.2014 10:51, schrieb Alon Bar-Lev: the ssh-keygen does not. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
For what it's worth I managed to get the ovirt-engine's public key from engine.cer using Ruby and turn it into a Puppet fact. Had to borrow some code from https://github.com/bensie/sshkey https://github.com/treydock/puppet-ovirt/blob/1.x/lib/facter/ovirt_engine_ssh_pubkey.rb Thanks for all the help Alon, now have semi-automated deployment of nodes :). Once 3.5 is released and the Foreman integration is in place, it will be much nicer. Thanks, - Trey On Fri, Aug 22, 2014 at 5:30 AM, Alon Bar-Lev alo...@redhat.com wrote: you are hijacking this thread... but anyway... please refer to the original question, how to easily convert X.509 certificate to SSH public key. the best method should avoid using the private key. newer ssh-keygen supports exactly that. - Original Message - From: Sven Kieske s.kie...@mittwald.de To: Alon Bar-Lev alo...@redhat.com Cc: users@ovirt.org Sent: Friday, August 22, 2014 1:24:17 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options well yeah, it does not generate pkcs#8 by default but you can easily convert existing keys via openssl: openssl pkcs8 -topk8 -v2 des3 \ -in test_rsa_key.old -passin 'pass:super secret passphrase' \ -out test_rsa_key -passout 'pass:super secret passphrase' see this page for more details: http://martin.kleppmann.com/2013/05/24/improving-security-of-ssh-private-keys.html newer ssh-keygen versions use PBKDF2 by default and not MD5 anymore. HTH Am 22.08.2014 10:51, schrieb Alon Bar-Lev: the ssh-keygen does not. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
I likely won't automate this yet, as a lot of what's coming in 3.5 seems to obsolete many things I was doing previously via Puppet. In particular the Foreman integration and the ability to add custom iptables rules to engine-config. Previous posts on the list made is seem like modifying IPTables could potentially make upgrades less reliable. Created a gist of a working series of commands based on Alon's example using the Host Deploy Protocol [1]. https://gist.github.com/treydock/570a776b5c160bca7c9c Curious , where is the public key used by the ovirt-engine stored? The one that is available using command=get-ssh-trust. Is there a way to query it from the engine? I'm thinking if it would be possible to create a custom Facter face that stores the value of that public key so easier to re-use and access for deployment. Thanks, - Trey [1] - http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 11:32 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 11:27:45 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? yes. you can download it via: $ curl 'http://engine/ovirt-engine/services/pki-resource?resource=engine-certificateformat=OPENSSH-PUBKEY' $ curl 'http://engine/ovirt-engine/services/host-register?version=1command=get-ssh-trust' probably better to use https and verify CA certificate fingerprint if you do that from host. Any examples of how to use the HostDeployProtocol [1]? I like the idea of using registration but haven't the slightest idea how to implement what's described in the docs [1]. I do recall seeing an article posted (searching email and can't find) that had a nice walk-through of how to use the oVirt API using browser tools. I'm unsure if this HostDeployProtocol would be done that way or via some other method. there are two apis, the formal rest-api that is exposed by the engine and can be accessed using any rest api tool or ovirt-engine-cli, ovirt-engine-sdk-java, ovirt-engine-sdk-python wrappers. I sent you a minimal example in previous message. and the host-deploy protocol[1], which should have been exposed in the rest-api, but for some reason I cannot understand it was not included in the public interface of the engine. the advantage of using the rest-api is that you can achieve full cycle using the protocol, the add host cycle is what you seek. the host-deploy protocol just register the host, but the sysadmin needs to approve the host via the ui (or via the rest api) before it is usable. Thanks, - Trey [1] http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 3:01 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 10:45:12 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Excellent, so installing 'ovirt-host-deploy' on each node then configuring the /etc/ovirt-host-deploy.conf.d files seems very automate-able, will see how it works in practice. you do not need to install the ovirt-host-deploy, just create the files. Regarding the actual host registration and getting the host added to ovirt-engine, are there other methods besides the API and the sdk? Would it be possible to configure the necessary ovirt-host-deploy.conf.d files then execute ovirt-host-deploy? I notice that running 'ovirt-host-deploy' wants to make whatever host executes it a ovir hypervisor but haven't yet run it all the way through as no server to test with at this time. There seems to be no --help or similar command line argument. you should not run host-deploy directly, but via the engine's process, either registration or add host as I replied previously. when base system is ready, you issue add host via api of engine or via ui, the other alternative is to register the host host the host-deploy protocol, and approve the host via api of engine or via ui. I'm sure this will all be more clear once I attempt the steps and run through the motions. Will try to find a system to test on so I'm ready once our new
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 9:41:03 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Sorry, I meant the SSH public key. Is that a file or in the database? I did a grep for the public key downloaded via the command=get-ssh-trust and found no files in /etc/ or /var/lib/ovirt-engine that matched. openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -m PKCS8 -f /dev/stdin - Trey On Thu, Aug 21, 2014 at 11:33 AM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 7:15:56 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options I likely won't automate this yet, as a lot of what's coming in 3.5 seems to obsolete many things I was doing previously via Puppet. In particular the Foreman integration and the ability to add custom iptables rules to engine-config. Previous posts on the list made is seem like modifying IPTables could potentially make upgrades less reliable. Created a gist of a working series of commands based on Alon's example using the Host Deploy Protocol [1]. https://gist.github.com/treydock/570a776b5c160bca7c9c Curious , where is the public key used by the ovirt-engine stored? The one that is available using command=get-ssh-trust. Is there a way to query it from the engine? I'm thinking if it would be possible to create a custom Facter face that stores the value of that public key so easier to re-use and access for deployment. /etc/pki/ovirt-engine/certs/engine.cer Thanks, - Trey [1] - http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 11:32 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 11:27:45 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? yes. you can download it via: $ curl 'http://engine/ovirt-engine/services/pki-resource?resource=engine-certificateformat=OPENSSH-PUBKEY' $ curl 'http://engine/ovirt-engine/services/host-register?version=1command=get-ssh-trust' probably better to use https and verify CA certificate fingerprint if you do that from host. Any examples of how to use the HostDeployProtocol [1]? I like the idea of using registration but haven't the slightest idea how to implement what's described in the docs [1]. I do recall seeing an article posted (searching email and can't find) that had a nice walk-through of how to use the oVirt API using browser tools. I'm unsure if this HostDeployProtocol would be done that way or via some other method. there are two apis, the formal rest-api that is exposed by the engine and can be accessed using any rest api tool or ovirt-engine-cli, ovirt-engine-sdk-java, ovirt-engine-sdk-python wrappers. I sent you a minimal example in previous message. and the host-deploy protocol[1], which should have been exposed in the rest-api, but for some reason I cannot understand it was not included in the public interface of the engine. the advantage of using the rest-api is that you can achieve full cycle using the protocol, the add host cycle is what you seek. the host-deploy protocol just register the host, but the sysadmin needs to approve the host via the ui (or via the rest api) before it is usable. Thanks, - Trey [1] http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 3:01 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Is there a method that works in EL6? $ openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -m PKCS8 -f /dev/stdin ssh-keygen: illegal option -- m $ openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -f /dev/stdin buffer_get_string_ret: bad string length 813826338 key_from_blob: can't read key type decode blob failed. I achieved somewhat similar result by doing the following, though likely is a security issue having something like Facter read from /etc/pki/ovirt-engine/keys $ ssh-keygen -y -f /etc/pki/ovirt-engine/keys/engine_id_rsa ssh-rsa PUBKEY Thanks, - Trey On Thu, Aug 21, 2014 at 1:44 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 9:41:03 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Sorry, I meant the SSH public key. Is that a file or in the database? I did a grep for the public key downloaded via the command=get-ssh-trust and found no files in /etc/ or /var/lib/ovirt-engine that matched. openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -m PKCS8 -f /dev/stdin - Trey On Thu, Aug 21, 2014 at 11:33 AM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 7:15:56 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options I likely won't automate this yet, as a lot of what's coming in 3.5 seems to obsolete many things I was doing previously via Puppet. In particular the Foreman integration and the ability to add custom iptables rules to engine-config. Previous posts on the list made is seem like modifying IPTables could potentially make upgrades less reliable. Created a gist of a working series of commands based on Alon's example using the Host Deploy Protocol [1]. https://gist.github.com/treydock/570a776b5c160bca7c9c Curious , where is the public key used by the ovirt-engine stored? The one that is available using command=get-ssh-trust. Is there a way to query it from the engine? I'm thinking if it would be possible to create a custom Facter face that stores the value of that public key so easier to re-use and access for deployment. /etc/pki/ovirt-engine/certs/engine.cer Thanks, - Trey [1] - http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 11:32 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 11:27:45 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? yes. you can download it via: $ curl 'http://engine/ovirt-engine/services/pki-resource?resource=engine-certificateformat=OPENSSH-PUBKEY' $ curl 'http://engine/ovirt-engine/services/host-register?version=1command=get-ssh-trust' probably better to use https and verify CA certificate fingerprint if you do that from host. Any examples of how to use the HostDeployProtocol [1]? I like the idea of using registration but haven't the slightest idea how to implement what's described in the docs [1]. I do recall seeing an article posted (searching email and can't find) that had a nice walk-through of how to use the oVirt API using browser tools. I'm unsure if this HostDeployProtocol would be done that way or via some other method. there are two apis, the formal rest-api that is exposed by the engine and can be accessed using any rest api tool or ovirt-engine-cli, ovirt-engine-sdk-java, ovirt-engine-sdk-python wrappers. I sent you a minimal example in previous message. and the host-deploy protocol[1], which should have been exposed in the rest-api, but for some reason I
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 10:55:28 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Is there a method that works in EL6? $ openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -m PKCS8 -f /dev/stdin ssh-keygen: illegal option -- m $ openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -f /dev/stdin buffer_get_string_ret: bad string length 813826338 key_from_blob: can't read key type decode blob failed. I achieved somewhat similar result by doing the following, though likely is a security issue having something like Facter read from /etc/pki/ovirt-engine/keys $ ssh-keygen -y -f /etc/pki/ovirt-engine/keys/engine_id_rsa ssh-rsa PUBKEY this is one option that use private key and unsupported artifact... better: openssl pkcs12 -in /etc/pki/ovirt-engine/keys/engine.p12 -nocerts -passin pass:mypass -nodes 2 /dev/null | ssh-keygen -y -f /dev/stdin I hope el6 will have newer openssh with support for the PKCS8 format. Thanks, - Trey On Thu, Aug 21, 2014 at 1:44 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 9:41:03 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Sorry, I meant the SSH public key. Is that a file or in the database? I did a grep for the public key downloaded via the command=get-ssh-trust and found no files in /etc/ or /var/lib/ovirt-engine that matched. openssl x509 -in /etc/pki/ovirt-engine/certs/engine.cer -noout -pubkey | ssh-keygen -i -m PKCS8 -f /dev/stdin - Trey On Thu, Aug 21, 2014 at 11:33 AM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Thursday, August 21, 2014 7:15:56 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options I likely won't automate this yet, as a lot of what's coming in 3.5 seems to obsolete many things I was doing previously via Puppet. In particular the Foreman integration and the ability to add custom iptables rules to engine-config. Previous posts on the list made is seem like modifying IPTables could potentially make upgrades less reliable. Created a gist of a working series of commands based on Alon's example using the Host Deploy Protocol [1]. https://gist.github.com/treydock/570a776b5c160bca7c9c Curious , where is the public key used by the ovirt-engine stored? The one that is available using command=get-ssh-trust. Is there a way to query it from the engine? I'm thinking if it would be possible to create a custom Facter face that stores the value of that public key so easier to re-use and access for deployment. /etc/pki/ovirt-engine/certs/engine.cer Thanks, - Trey [1] - http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 11:32 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 11:27:45 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? yes. you can download it via: $ curl 'http://engine/ovirt-engine/services/pki-resource?resource=engine-certificateformat=OPENSSH-PUBKEY' $ curl 'http://engine/ovirt-engine/services/host-register?version=1command=get-ssh-trust' probably better to use
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
It seems that you should be able to change it: http://www.supermicro.com/support/faqs/faq.cfm?faq=15869 Did not try that myself. Not sure it's better than simply managing vdsm.id manually. -- Didi - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Joop jvdw...@xs4all.nl Cc: users users@ovirt.org Sent: Tuesday, August 5, 2014 10:17:06 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Yep, these are Supermicro servers. # facter serialnumber 1234567890 On Tue, Aug 5, 2014 at 2:07 PM, Joop jvdw...@xs4all.nl wrote: On 5-8-2014 20:54, Alon Bar-Lev wrote: * ensure vdsm.conf exists * Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this) why? you machine should have valid bios uuid. There are manufactures that don't set a unique bios uuid. Supermicro is one of them. I had a RFE but that has been closed but I feel that it should be reopened and implemented. The change is minimal and guarantees that there is a unique uuid. Regards, Joop ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Hi, Do you actually use puppet over ovirt-node? This is unsupported. Regards, Alon - Original Message - From: ybronhei ybron...@redhat.com To: Trey Dockendorf treyd...@gmail.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 8:32:04 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos, and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). What I've noticed is that after step #3, after a host is added by ovirt-engine, the vdsm.conf file is reset to default and I have to reapply Puppet before it can be used as the one of my Data Storage Domains requires iSER (not available over TCP). What would be the workflow using ovirt-host-deploy? Thus far I've had to piece together my workflow based on the documentation and filling in blanks where possible since I do require customizations to vdsm.conf and the documented workflow of adding a host via web UI does not allow for such customization. Thanks, - Trey [1] - https://github.com/treydock/puppet-ovirt (README not fully updated as still working out how to use Puppet with oVirt) On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: ybronhei ybron...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 9:36:24 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos and should stop here.. , and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. you can create a file /etc/ovirt-host-deploy.conf.d/40-xxx.conf --- VDSM_CONFIG/section/key=str:content --- this will create a proper vdsm.conf when host-deploy is initiated. you should now use the rest api to initiate host-deploy. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). right, but you should let this process install packages and manage configuration. What I've noticed is that after step #3, after a host is added by ovirt-engine, the vdsm.conf file is reset to default and I have to reapply Puppet before it can be used as the one of my Data Storage Domains requires iSER (not available over TCP). right, see above. What would be the workflow using ovirt-host-deploy? Thus far I've had to piece together my workflow based on the documentation and filling in blanks where possible since I do require customizations to vdsm.conf and the documented workflow of adding a host via web UI does not allow for such customization. Thanks, - Trey [1] - https://github.com/treydock/puppet-ovirt (README not fully updated as still working out how to use Puppet with oVirt) On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On Tue, Aug 5, 2014 at 12:37 PM, Alon Bar-Lev alo...@redhat.com wrote: Hi, Do you actually use puppet over ovirt-node? This is unsupported. Regards, Alon I use Puppet to configure everything on the system and some of those things conflict with changes made by ovirt-engine when adding a node so I've moved to managing those changes in an ovirt Puppet module. When you refer to ovirt-node your refering to the pre-built ISO images? AFAIK the ovirt-node is not an option for me as I use Infiniband to do all storage connections. Currently NFS is done using IPoIB (NFS over RDMA was crashing so I am not pursuing at the moment) and iSCSI is done using iSER. At some point in time I'd like to use IB interfaces in my guests utilizing SR-IOV which my Mellanox cards support, but I'm postponing that project till after our HPC cluster's upgraded :) Right now the Puppet module does the following: * Configures firewall rules just like what is done by ovirt (with exception of a few not supported by puppetlabs-firewall, so I override global directive that purges unknown firewall rules) ** This is necessary as I have to add other Firewall rules, such as Zabbix * Exports /etc/hosts entry for the node that is collected by ovirt-engine host so that if DNS goes down ovirt-engine does not lose access to ovirtmgmt interfaces * Install yum repos for ovirt * install vdsm * ensure vdsm.conf exists * Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this) * Ensures vdsmd is running and will start at boot * Ensures vdsm sudo rules are present. * Manages default vdsm.conf configurations as a Puppet type, vdsm_config, rather than managing file contents via template (allows for purging unmanaged entries also) A lot of the above is handled by ovirt already, but in the past customizations were not possible (firewall rules, vdsm.conf entries, etc) so if I was going to have to manage those separately I wanted them in Puppet :) Now that I'm aware of ovirt-host-deploy and have seen the potential of using vdsm-reg in Puppet, I'm curious what is the right way to automate node deployment's in oVirt. Ideally I could still use Puppet to configure the method or fill in the gaps for customizations that are needed (ie enabling iSER). I'd be glad to know what the recommended method for automating ovirt would be, and would be happy to refactor my module in hopes it would offer other Puppet users a quick-start way to deploy oVirt while still doing things the ovirt way. If the right way is using the ovirt-node images then I'd like to know what customizations are possible on those images and so on. oVirt is moving very rapidly and despite using ovirt for a long time I'm still learning new things about it almost daily, so forgive any of my assumptions above that may be wrong :). Thanks, - Trey - Original Message - From: ybronhei ybron...@redhat.com To: Trey Dockendorf treyd...@gmail.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 8:32:04 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, ybronhei ybron...@redhat.com Sent: Tuesday, August 5, 2014 9:50:04 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:37 PM, Alon Bar-Lev alo...@redhat.com wrote: Hi, Do you actually use puppet over ovirt-node? This is unsupported. Regards, Alon I use Puppet to configure everything on the system and some of those things conflict with changes made by ovirt-engine when adding a node so I've moved to managing those changes in an ovirt Puppet module. When you refer to ovirt-node your refering to the pre-built ISO images? AFAIK the ovirt-node is not an option for me as I use Infiniband to do all storage connections. Currently NFS is done using IPoIB (NFS over RDMA was crashing so I am not pursuing at the moment) and iSCSI is done using iSER. At some point in time I'd like to use IB interfaces in my guests utilizing SR-IOV which my Mellanox cards support, but I'm postponing that project till after our HPC cluster's upgraded :) Right now the Puppet module does the following: * Configures firewall rules just like what is done by ovirt (with exception of a few not supported by puppetlabs-firewall, so I override global directive that purges unknown firewall rules) this should be done at engine side, using engine-config, seek IPTablesConfig or IPTablesConfigSiteCustom in 3.5. ** This is necessary as I have to add other Firewall rules, such as Zabbix * Exports /etc/hosts entry for the node that is collected by ovirt-engine host so that if DNS goes down ovirt-engine does not lose access to ovirtmgmt interfaces why not just have a backup for dns? * Install yum repos for ovirt this should not be done within ovirt-node. * install vdsm this should be done by host-deploy. * ensure vdsm.conf exists * Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this) why? you machine should have valid bios uuid. * Ensures vdsmd is running and will start at boot * Ensures vdsm sudo rules are present. * Manages default vdsm.conf configurations as a Puppet type, vdsm_config, rather than managing file contents via template (allows for purging unmanaged entries also) the above should be done via host-deploy. A lot of the above is handled by ovirt already, but in the past customizations were not possible (firewall rules, vdsm.conf entries, etc) so if I was going to have to manage those separately I wanted them in Puppet :) Now that I'm aware of ovirt-host-deploy and have seen the potential of using vdsm-reg in Puppet, I'm curious what is the right way to automate node deployment's in oVirt. Ideally I could still use Puppet to configure the method or fill in the gaps for customizations that are needed (ie enabling iSER). I'd be glad to know what the recommended method for automating ovirt would be, and would be happy to refactor my module in hopes it would offer other Puppet users a quick-start way to deploy oVirt while still doing things the ovirt way. If the right way is using the ovirt-node images then I'd like to know what customizations are possible on those images and so on. oVirt is moving very rapidly and despite using ovirt for a long time I'm still learning new things about it almost daily, so forgive any of my assumptions above that may be wrong :). ovirt-node is not flexible as standard host, if you require sssd and misc packages I suggest you stick with generic host. Thanks, - Trey - Original Message - From: ybronhei ybron...@redhat.com To: Trey Dockendorf treyd...@gmail.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 8:32:04 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf For #2b I have a few questions * The name of the .conf file is simply for sorting and labeling/organization, it has not functional impact on what those overrides apply to? * That file is managed on the ovirt-engine server, not the actual nodes? * Is there any way to apply overrides to specific hosts? For example if I have some hosts that require a config and others that don't, how would I separate those *.conf files? This is more theoretical as right now my setup is common across all nodes. For #3...the implementation of API calls from within Puppet is a challenge and one I can't tackle yet, but definitely will make it a goal for the future. In the mean time, what's the manual way to initiate host-deploy? Is there a CLI command that would have the same result as an API call or is the recommended way to perform the API call manually (ie curl)? Thanks! - Trey On Tue, Aug 5, 2014 at 1:45 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: ybronhei ybron...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 9:36:24 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos and should stop here.. , and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. you can create a file /etc/ovirt-host-deploy.conf.d/40-xxx.conf --- VDSM_CONFIG/section/key=str:content --- this will create a proper vdsm.conf when host-deploy is initiated. you should now use the rest api to initiate host-deploy. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). right, but you should let this process install packages and manage configuration. What I've noticed is that after step #3, after a host is added by ovirt-engine, the vdsm.conf file is reset to default and I have to reapply Puppet before it can be used as the one of my Data Storage Domains requires iSER (not available over TCP). right, see above. What would be the workflow using ovirt-host-deploy? Thus far I've had to piece together my workflow based on the documentation and filling in blanks where possible since I do require customizations to vdsm.conf and the documented workflow of adding a host via web UI does not allow for such customization. Thanks, - Trey [1] - https://github.com/treydock/puppet-ovirt (README not fully updated as still working out how to use Puppet with oVirt) On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On 5-8-2014 20:54, Alon Bar-Lev wrote: * ensure vdsm.conf exists * Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this) why? you machine should have valid bios uuid. There are manufactures that don't set a unique bios uuid. Supermicro is one of them. I had a RFE but that has been closed but I feel that it should be reopened and implemented. The change is minimal and guarantees that there is a unique uuid. Regards, Joop ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com Sent: Tuesday, August 5, 2014 10:01:14 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf you can have any # of files with any prefix :)) For #2b I have a few questions * The name of the .conf file is simply for sorting and labeling/organization, it has not functional impact on what those overrides apply to? right. * That file is managed on the ovirt-engine server, not the actual nodes? currently on the host, in future we will provide a method to add this to engine database[1] [1] http://gerrit.ovirt.org/#/c/27064/ * Is there any way to apply overrides to specific hosts? For example if I have some hosts that require a config and others that don't, how would I separate those *.conf files? This is more theoretical as right now my setup is common across all nodes. the poppet module can put whatever required on each host. For #3...the implementation of API calls from within Puppet is a challenge and one I can't tackle yet, but definitely will make it a goal for the future. In the mean time, what's the manual way to initiate host-deploy? Is there a CLI command that would have the same result as an API call or is the recommended way to perform the API call manually (ie curl)? well, you can register host using the following protocol[1], but it is difficult to do this securely, what you actually need is to establish ssh trust for root with engine key then register. you can also use the register command using curl by something like (I have not checked): https://admin%40internal:password@engine/ovirt-engine/api/hosts --- ?xml version=1.0 encoding=UTF-8 standalone=yes? host namehost1/name addressdns/address ssh authentication_methodpublickey/authentication_method /ssh cluster id=cluster-uuid/ /host --- you can also use the ovirt-engine-sdk-python package: --- import ovirtsdk.api import ovirtsdk.xml sdk = ovirtsdk.api.API( url='https://host/ovirt-engine/api', username='admin@internal', password='password', insecure=True, ) sdk.hosts.add( ovirtsdk.xml.params.Host( name='host1', address='host1', cluster=engine_api.clusters.get( 'cluster' ), ssh=self._ovirtsdk_xml.params.SSH( authentication_method='publickey', ), ) ) --- [1] http://www.ovirt.org/Features/HostDeployProtocol Thanks! - Trey On Tue, Aug 5, 2014 at 1:45 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: ybronhei ybron...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 9:36:24 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos and should stop here.. , and installs packages. Part of my Puppet deployment
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Excellent, so installing 'ovirt-host-deploy' on each node then configuring the /etc/ovirt-host-deploy.conf.d files seems very automate-able, will see how it works in practice. Regarding the actual host registration and getting the host added to ovirt-engine, are there other methods besides the API and the sdk? Would it be possible to configure the necessary ovirt-host-deploy.conf.d files then execute ovirt-host-deploy? I notice that running 'ovirt-host-deploy' wants to make whatever host executes it a ovir hypervisor but haven't yet run it all the way through as no server to test with at this time. There seems to be no --help or similar command line argument. I'm sure this will all be more clear once I attempt the steps and run through the motions. Will try to find a system to test on so I'm ready once our new servers arrive. Thanks, - Trey On Tue, Aug 5, 2014 at 2:23 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com Sent: Tuesday, August 5, 2014 10:01:14 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf you can have any # of files with any prefix :)) For #2b I have a few questions * The name of the .conf file is simply for sorting and labeling/organization, it has not functional impact on what those overrides apply to? right. * That file is managed on the ovirt-engine server, not the actual nodes? currently on the host, in future we will provide a method to add this to engine database[1] [1] http://gerrit.ovirt.org/#/c/27064/ * Is there any way to apply overrides to specific hosts? For example if I have some hosts that require a config and others that don't, how would I separate those *.conf files? This is more theoretical as right now my setup is common across all nodes. the poppet module can put whatever required on each host. For #3...the implementation of API calls from within Puppet is a challenge and one I can't tackle yet, but definitely will make it a goal for the future. In the mean time, what's the manual way to initiate host-deploy? Is there a CLI command that would have the same result as an API call or is the recommended way to perform the API call manually (ie curl)? well, you can register host using the following protocol[1], but it is difficult to do this securely, what you actually need is to establish ssh trust for root with engine key then register. you can also use the register command using curl by something like (I have not checked): https://admin%40internal:password@engine/ovirt-engine/api/hosts --- ?xml version=1.0 encoding=UTF-8 standalone=yes? host namehost1/name addressdns/address ssh authentication_methodpublickey/authentication_method /ssh cluster id=cluster-uuid/ /host --- you can also use the ovirt-engine-sdk-python package: --- import ovirtsdk.api import ovirtsdk.xml sdk = ovirtsdk.api.API( url='https://host/ovirt-engine/api', username='admin@internal', password='password', insecure=True, ) sdk.hosts.add( ovirtsdk.xml.params.Host( name='host1', address='host1', cluster=engine_api.clusters.get( 'cluster' ), ssh=self._ovirtsdk_xml.params.SSH( authentication_method='publickey', ), ) ) --- [1] http://www.ovirt.org/Features/HostDeployProtocol Thanks! - Trey On Tue, Aug 5, 2014 at 1:45 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: ybronhei ybron...@redhat.com Cc: users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Alon Bar-Lev alo...@redhat.com Sent: Tuesday, August 5, 2014 9:36:24 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 10:45:12 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Excellent, so installing 'ovirt-host-deploy' on each node then configuring the /etc/ovirt-host-deploy.conf.d files seems very automate-able, will see how it works in practice. you do not need to install the ovirt-host-deploy, just create the files. Regarding the actual host registration and getting the host added to ovirt-engine, are there other methods besides the API and the sdk? Would it be possible to configure the necessary ovirt-host-deploy.conf.d files then execute ovirt-host-deploy? I notice that running 'ovirt-host-deploy' wants to make whatever host executes it a ovir hypervisor but haven't yet run it all the way through as no server to test with at this time. There seems to be no --help or similar command line argument. you should not run host-deploy directly, but via the engine's process, either registration or add host as I replied previously. when base system is ready, you issue add host via api of engine or via ui, the other alternative is to register the host host the host-deploy protocol, and approve the host via api of engine or via ui. I'm sure this will all be more clear once I attempt the steps and run through the motions. Will try to find a system to test on so I'm ready once our new servers arrive. Thanks, - Trey On Tue, Aug 5, 2014 at 2:23 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com Sent: Tuesday, August 5, 2014 10:01:14 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf you can have any # of files with any prefix :)) For #2b I have a few questions * The name of the .conf file is simply for sorting and labeling/organization, it has not functional impact on what those overrides apply to? right. * That file is managed on the ovirt-engine server, not the actual nodes? currently on the host, in future we will provide a method to add this to engine database[1] [1] http://gerrit.ovirt.org/#/c/27064/ * Is there any way to apply overrides to specific hosts? For example if I have some hosts that require a config and others that don't, how would I separate those *.conf files? This is more theoretical as right now my setup is common across all nodes. the poppet module can put whatever required on each host. For #3...the implementation of API calls from within Puppet is a challenge and one I can't tackle yet, but definitely will make it a goal for the future. In the mean time, what's the manual way to initiate host-deploy? Is there a CLI command that would have the same result as an API call or is the recommended way to perform the API call manually (ie curl)? well, you can register host using the following protocol[1], but it is difficult to do this securely, what you actually need is to establish ssh trust for root with engine key then register. you can also use the register command using curl by something like (I have not checked): https://admin%40internal:password@engine/ovirt-engine/api/hosts --- ?xml version=1.0 encoding=UTF-8 standalone=yes? host namehost1/name addressdns/address ssh authentication_methodpublickey/authentication_method /ssh cluster id=cluster-uuid/ /host --- you can also use the ovirt-engine-sdk-python package: --- import ovirtsdk.api import ovirtsdk.xml sdk = ovirtsdk.api.API( url='https://host/ovirt-engine/api', username='admin@internal', password='password', insecure=True, ) sdk.hosts.add( ovirtsdk.xml.params.Host( name='host1', address='host1', cluster=engine_api.clusters.get( 'cluster
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? Any examples of how to use the HostDeployProtocol [1]? I like the idea of using registration but haven't the slightest idea how to implement what's described in the docs [1]. I do recall seeing an article posted (searching email and can't find) that had a nice walk-through of how to use the oVirt API using browser tools. I'm unsure if this HostDeployProtocol would be done that way or via some other method. Thanks, - Trey [1] http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 3:01 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 10:45:12 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Excellent, so installing 'ovirt-host-deploy' on each node then configuring the /etc/ovirt-host-deploy.conf.d files seems very automate-able, will see how it works in practice. you do not need to install the ovirt-host-deploy, just create the files. Regarding the actual host registration and getting the host added to ovirt-engine, are there other methods besides the API and the sdk? Would it be possible to configure the necessary ovirt-host-deploy.conf.d files then execute ovirt-host-deploy? I notice that running 'ovirt-host-deploy' wants to make whatever host executes it a ovir hypervisor but haven't yet run it all the way through as no server to test with at this time. There seems to be no --help or similar command line argument. you should not run host-deploy directly, but via the engine's process, either registration or add host as I replied previously. when base system is ready, you issue add host via api of engine or via ui, the other alternative is to register the host host the host-deploy protocol, and approve the host via api of engine or via ui. I'm sure this will all be more clear once I attempt the steps and run through the motions. Will try to find a system to test on so I'm ready once our new servers arrive. Thanks, - Trey On Tue, Aug 5, 2014 at 2:23 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com Sent: Tuesday, August 5, 2014 10:01:14 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf you can have any # of files with any prefix :)) For #2b I have a few questions * The name of the .conf file is simply for sorting and labeling/organization, it has not functional impact on what those overrides apply to? right. * That file is managed on the ovirt-engine server, not the actual nodes? currently on the host, in future we will provide a method to add this to engine database[1] [1] http://gerrit.ovirt.org/#/c/27064/ * Is there any way to apply overrides to specific hosts? For example if I have some hosts that require a config and others that don't, how would I separate those *.conf files? This is more theoretical as right now my setup is common across all nodes. the poppet module can put whatever required on each host. For #3...the implementation of API calls from within Puppet is a challenge and one I can't tackle yet, but definitely will make it a goal for the future. In the mean time, what's the manual way to initiate host-deploy? Is there a CLI command that would have the same result as an API call or is the recommended way to perform the API call manually (ie curl)? well, you can register host using the following protocol[1], but it is difficult to do this securely, what you actually need is to establish ssh trust for root with engine key then register. you can also use the register command using curl by something like (I have not checked): https://admin%40internal:password@engine/ovirt-engine/api/hosts --- ?xml version=1.0 encoding=UTF-8 standalone=yes? host
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On 08/05/2014 09:36 PM, Trey Dockendorf wrote: On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos, and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). just wondering (and i may have missed this in the thread) - if you want bare metal provisioning and foreman, why not just use the new 3.5 integration doing that: http://www.ovirt.org/Features/AdvancedForemanIntegration it will call foreman, do bare metal provision with the hostgroup you chose, then foreman will call add host in the engine on your behalf? (doesn't limit you from either extending ovirt-host-deploy plugins, or using more puppet modules via the hostgroup). ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On Tue, Aug 5, 2014 at 3:31 PM, Itamar Heim ih...@redhat.com wrote: On 08/05/2014 09:36 PM, Trey Dockendorf wrote: On Tue, Aug 5, 2014 at 12:32 PM, ybronhei ybron...@redhat.com wrote: Hey, Just noticed something that I forgot about.. before filing new BZ, see in ovirt-host-deploy README.environment [1] the section: VDSM/configOverride(bool) [True] Override vdsm configuration file. changing it to false will keep your vdsm.conf file as is after deploying the host again (what happens after node upgrade) [1] https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment please check if that what you meant.. Thanks, Yaniv Bronhaim. I was unaware of that package. I will check that out as that seems to be what I am looking for. I have not filed this in BZ and will hold off pending ovirt-host-deploy. If you feel a BZ is still necessary then please do file one and I would be happy to provide input if it would help. Right now this is my workflow. 1. Foreman provisions bare-metal server with CentOS 6.5 2. Once provisioned and system rebooted Puppet applies puppet-ovirt [1] module that adds the necessary yum repos, and installs packages. Part of my Puppet deployment is basic things like sudo management (vdsm's sudo is account for), sssd configuration, and other aspects that are needed by every system in my infrastructure. Part of the ovirt::node Puppet class is managing vdsm.conf, and in my case that means ensuring iSER is enabled for iSCSI over IB. 3. Once host is online and has had the full Puppet catalog applied I log into ovirt-engine web interface and add those host (pulling it's data via the Foreman provider). just wondering (and i may have missed this in the thread) - if you want bare metal provisioning and foreman, why not just use the new 3.5 integration doing that: http://www.ovirt.org/Features/AdvancedForemanIntegration it will call foreman, do bare metal provision with the hostgroup you chose, then foreman will call add host in the engine on your behalf? (doesn't limit you from either extending ovirt-host-deploy plugins, or using more puppet modules via the hostgroup). I was actually looking forward to using those features but haven't yet had a chance to test 3.5 as I have no spare hardware at the moment. The Foreman Integration is definitely on my radar as we rely very heavily on Foreman. Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
- Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 11:27:45 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Thanks for clarifying, makes sense now. The public key trust needed for registration, is that the same key that would be used when adding host via UI? yes. you can download it via: $ curl 'http://engine/ovirt-engine/services/pki-resource?resource=engine-certificateformat=OPENSSH-PUBKEY' $ curl 'http://engine/ovirt-engine/services/host-register?version=1command=get-ssh-trust' probably better to use https and verify CA certificate fingerprint if you do that from host. Any examples of how to use the HostDeployProtocol [1]? I like the idea of using registration but haven't the slightest idea how to implement what's described in the docs [1]. I do recall seeing an article posted (searching email and can't find) that had a nice walk-through of how to use the oVirt API using browser tools. I'm unsure if this HostDeployProtocol would be done that way or via some other method. there are two apis, the formal rest-api that is exposed by the engine and can be accessed using any rest api tool or ovirt-engine-cli, ovirt-engine-sdk-java, ovirt-engine-sdk-python wrappers. I sent you a minimal example in previous message. and the host-deploy protocol[1], which should have been exposed in the rest-api, but for some reason I cannot understand it was not included in the public interface of the engine. the advantage of using the rest-api is that you can achieve full cycle using the protocol, the add host cycle is what you seek. the host-deploy protocol just register the host, but the sysadmin needs to approve the host via the ui (or via the rest api) before it is usable. Thanks, - Trey [1] http://www.ovirt.org/Features/HostDeployProtocol On Tue, Aug 5, 2014 at 3:01 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com, Oved Ourfali ov...@redhat.com Sent: Tuesday, August 5, 2014 10:45:12 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Excellent, so installing 'ovirt-host-deploy' on each node then configuring the /etc/ovirt-host-deploy.conf.d files seems very automate-able, will see how it works in practice. you do not need to install the ovirt-host-deploy, just create the files. Regarding the actual host registration and getting the host added to ovirt-engine, are there other methods besides the API and the sdk? Would it be possible to configure the necessary ovirt-host-deploy.conf.d files then execute ovirt-host-deploy? I notice that running 'ovirt-host-deploy' wants to make whatever host executes it a ovir hypervisor but haven't yet run it all the way through as no server to test with at this time. There seems to be no --help or similar command line argument. you should not run host-deploy directly, but via the engine's process, either registration or add host as I replied previously. when base system is ready, you issue add host via api of engine or via ui, the other alternative is to register the host host the host-deploy protocol, and approve the host via api of engine or via ui. I'm sure this will all be more clear once I attempt the steps and run through the motions. Will try to find a system to test on so I'm ready once our new servers arrive. Thanks, - Trey On Tue, Aug 5, 2014 at 2:23 PM, Alon Bar-Lev alo...@redhat.com wrote: - Original Message - From: Trey Dockendorf treyd...@gmail.com To: Alon Bar-Lev alo...@redhat.com Cc: ybronhei ybron...@redhat.com, users users@ovirt.org, Fabian Deutsch fabi...@redhat.com, Dan Kenigsberg dan...@redhat.com, Itamar Heim ih...@redhat.com, Douglas Landgraf dougsl...@redhat.com Sent: Tuesday, August 5, 2014 10:01:14 PM Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options Ah, thank you for the input! Just so I'm not spending time implementing the wrong changes, let me confirm I understand your comments. 1) Deploy host with Foreman 2) Apply Puppet catalog including ovirt Puppet module 3) Initiate host-deploy via rest API In the ovirt module the following takes place: 2a) Add yum repos 2b) Manage /etc/ovirt-host-deploy.conf.d/40-xxx.conf
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
I'll file BZ. As far as I can recall this has been an issue since 3.3.x as I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Proper way to change and persist vdsm configuration options
On 08/05/2014 08:12 AM, Trey Dockendorf wrote: I'll file BZ. As far as I can recall this has been an issue since 3.3.x as please send a link to the BZ, or assign me there I have been using Puppet to modify values and have had to rerun Puppet after installing a node via GUI and when performing update from GUI. Given that it has occurred when VDSM version didn't change on the node it seems likely to be something being done by Python code that bootstraps a node and performs the other tasks. I won't have any systems available to test with for a few days. New hardware specifically for our oVirt deployment is on order so should be able to more thoroughly debug and capture logs at that time. Would using vdsm-reg be a better solution for adding new nodes? I only tried using vdsm-reg once and it went very poorly...lots of missing dependencies not pulled in from yum install I had to install manually via yum. Then the node was auto added to newest cluster with no ability to change the cluster. Be happy to debug that too if there's some docs that outline the expected behavior. Using vdsm-reg or something similar seems like a better fit for puppet deployed nodes, as opposed to requiring GUI steps to add the node. please consult Fabian\Douglas on using vdsm-reg, though, I don't think it will do any change. after vdsm-reg engine still runs host-deploy process and this might overrides your vdsm.conf (which I need to test) Thanks - Trey On Aug 4, 2014 5:53 AM, ybronhei ybron...@redhat.com wrote: On 07/31/2014 01:28 AM, Trey Dockendorf wrote: I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Hey, vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade. other related conf files (such as libvirtd.conf) might be overrided to keep defaults configurations for vdsm. but vdsm.conf should persist with user's modification. from my check, regular yum upgrade doesn't touch vdsm.conf Douglas can you verify that with node upgrade? might be specific to that flow.. Trey, can file a bugzilla on that and describe your steps there? Thanks Yaniv Bronhaim, Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Yaniv Bronhaim. -- Yaniv Bronhaim. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] Proper way to change and persist vdsm configuration options
I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM installed. I am using iSER to do iSCSI over RDMA and to make that work I have to modify /etc/vdsm/vdsm.conf to include the following: [irs] iscsi_default_ifaces = iser,default I've noticed that any time I upgrade a node from the engine web interface that changes to vdsm.conf are wiped out. I don't know if this is being done by the configuration code or by the vdsm package. Is there a more reliable way to ensure changes to vdsm.conf are NOT removed automatically? Thanks, - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
