Re: [Engine-devel] Cloud-Init integration
On 06/16/2013 05:08 PM, Mike Kolesnik wrote: - Original Message - On Thu, Mar 28, 2013 at 07:35:23PM -0400, Greg Padgett wrote: Hi Everyone, I'd like to propose a feature we've been doing some investigation into, which is to integrate cloud-init support into oVirt. Cloud-init is used to help provision new Linux systems by setting the hostname, ip, ssh keys, timezone, injecting files, and more. It's used by OpenStack (amongst others) now, and has a lot of features that may be helpful to our users. Details are still evolving, but for more info please see the wiki page: http://www.ovirt.org/Features/Cloud-Init_Integration All feedback is welcome! All feedback? Even if it's 3 months too late? I'll have to be more careful next time to specify a time limit :) But really, thank you both for the comments. Feel free to add them to a nice to have list on the wiki page, or I can add it. Replies inline... If so, then here's a few comments: 1. oVirt-3.2 completely supports IPv6 within guests. It would be nice to carry this on to cloud-init, by allowing to set the IPv6 address of the guest. (or are we happy with the auto configured ipv6 addresses?) It would, but unfortunately cloud-init doesn't yet translate ipv6 fields from /etc/network/interfaces (its chosen networking input format) to ifcfg files. Now that you mention it, it doesn't add IPV6INIT=yes, either. These are things we can add--to both oVirt and cloud-init, I guess at a later time. 2. I think that the GUI used for setting IP addresses should be immitated here. It allows Static/DHCP/None, and disables the irrelvant fields when DHCP/None is selected. It's close: there is a checkbox for dhcp, and if selected it will hide the non-relevant fields. Additionally, you should consider showing the vNIC name and not eth0 etc. IIUC udev rules are rather unpredictable in this regard and could give your vNIC a different name on different VM instances (probably by MAC address and/or PCI address). Either way, I think it's less confusing to refer to the vNIC itself. Similarly to the above, I'm not sure how much cloud-init supports with respect to vNICs. The name is used as the /etc/network/interfaces adapter name and the ifcfg-* suffix (per distro), so I guess if you plumbed everything around that in the image and could finish the setup with just the interfaces/ifcfg config file then it would work as-is. If that isn't adequate (I haven't looked or tested) then we may need to consider submitting a patch to add this to cloud-init. 3. Is Support custom volume label for vm payloads still on the TODO list? Note: F19's dosfstools has renamed mkfs.msdos to mkfs.fat. This means that creating a virtual floppy with a payload currently does not work there (it's a tiny fix, for sure). It's implemented. Of course, it still uses mkfs.msdos. 4. I do not see ovirt-guest-agent mentioned in the feaure page. It is the obvious channel to report that cloud-init is Done to Engine. You're right, and this is one part of the feature that hasn't been done. It may also require some work on cloud-init, or for us to use a different input format (i.e. a mime-formatted sequence instead of vanilla config-drive-2). It would be great to add this, though my time to do that now is a bit limited. 5. When we come to implement auto-generate of system ssh key, we may want to install a virtio-rng device in our VMs, to ensure that the keys are not too easy to guess. (or create the key in the host and inject it to the guest) Good idea. I'm not too familiar with virtio-rng, but if the image can be configured to use it then the regeneration should follow suit. So, not a limitation right now but not as easy as it could be. It's another case of needing either a cloud-init patch for support or (I'm thinking more likely in this case) some scripting and using a mime-formatted input to cloud-init. 6. Is this going to be supported on template/instance type level? Probably static IP is not wise on a template, but the other options seem like they would be the same for most VMs from the same template/instance. Today it's just on Run VM Once but once we can persist the cloud-init options I don't see why we wouldn't want to allow persisting at least at least some of the fields for templates/instance types. 7. Don't forget backwards compatibility considerations for the engine-VDSM communication, if you're using APIs that aren't available in older VDSM versions. Thanks, it does rely on some new vdsm API features. Dan. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Cloud-Init integration
Hi Itamar, On 04/20/2013 04:52 PM, Itamar Heim wrote: On 03/29/2013 01:35 AM, Greg Padgett wrote: Hi Everyone, I'd like to propose a feature we've been doing some investigation into, which is to integrate cloud-init support into oVirt. Cloud-init is used to help provision new Linux systems by setting the hostname, ip, ssh keys, timezone, injecting files, and more. It's used by OpenStack (amongst others) now, and has a lot of features that may be helpful to our users. Details are still evolving, but for more info please see the wiki page: http://www.ovirt.org/Features/Cloud-Init_Integration All feedback is welcome! a few questions: - are you planning to save the info in the db by field, or as a single blob? maybe a better questions is are you going to persist it at all? No persistence yet, but when that time comes I'd prefer to store the fields (either separately or a standard serialized format), rather than storing the generated blob. Easier to retrieve/modify. - i'd be careful before passing any passwords (page mentions root password) - you'd need to not persist it unecrypted, identify it and clean it from all logs, etc. I've looked into this a little: cloud-init will accept a password that's been crypt(3)'ed, and Apache Commons Codec has a Crypt API that will do this--but not in our version, it's new in 1.7. So, still considering alternatives. - hostname - should just assume the vm name? Sure, how about pre-populating the field? The user would see that we're setting the hostname and be able to change it if desired. - timezone - is that different than the windows one? for a windows guest as well? AFAICT, a healthy subset of timezones can be mapped cleanly between Linux/Java/Windows representations, and the Java timezone ids look to be a fairly good match for the timezone files used in Linux. I hope we don't have to show different timezone selections based on OS type, but not sure yet. (Should know soon, I'm looking into all this now.) Thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Cloud-Init integration
Hi Everyone, I'd like to propose a feature we've been doing some investigation into, which is to integrate cloud-init support into oVirt. Cloud-init is used to help provision new Linux systems by setting the hostname, ip, ssh keys, timezone, injecting files, and more. It's used by OpenStack (amongst others) now, and has a lot of features that may be helpful to our users. Details are still evolving, but for more info please see the wiki page: http://www.ovirt.org/Features/Cloud-Init_Integration All feedback is welcome! Thanks, Greg ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] DB Schema Reports
On 02/22/2013 06:05 AM, Eli Mesika wrote: - Original Message - From: Libor Spevak lspe...@redhat.com To: engine-devel@ovirt.org, rhev-de...@redhat.com, in...@ovirt.org Sent: Friday, February 22, 2013 10:02:55 AM Subject: DB Schema Reports Hi, let me announce new database schema reports based on the SchemaSpy Java library available. Thanks to infra group for support, especially to David Caro, who set up Mr. Jenkins's jobs. Great job ! +1, this will be very useful! Eli, is the great diagram you sent out in early December on the wiki as well? We should add a link to it from the wiki page below. oVirt-Engine: http://resources.ovirt.org/dbreports/latest/engine/public/index.html oVirt-DWH: http://resources.ovirt.org/dbreports/latest/dwh/public/index.html Wiki: http://www.ovirt.org/DB_model Regards, Libor ___ Infra mailing list in...@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra ___ Infra mailing list in...@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] 3.2 features for release notes
On 01/24/2013 10:30 AM, Itamar Heim wrote: On 23/01/2013 23:47, Cheryn Tan wrote: Hi all, I'm preparing the oVirt 3.2 release notes, starting with the features listed here: http://www.ovirt.org/OVirt_3.2_release-management#Features Can you please go through the list, and reply to this thread if: a) There are features being released for 3.2 which should be documented, but are not on the list; or b) Some of the planned features on this list are not being released for 3.2; or c) Some of the features on this list are being released for 3.2, but should not be documented. For features which aren't on the wiki, please send a short description or Bugzilla numbers of each feature. Please reply no later than Jan 28. Thank you very much for your help! Cheryn ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel I think most of these made it to 3.2, i'll point fingers at others to give more details... (doesn't mean others weren't involved in them, just picked someone to give the details for release notes). SSIA: Add support for Opteron G5 (Seoul) Add Support for Intel Haswell CPU windows 8/12 - add os type windows 8/12 - default in user portal should be rdp (till we have spice support...) expand tree by default with list of DCs federico: storage live migration tomas: Configure Smartcard for VM including Portal support Allow e1000 to be selected as nic type for Windows VM dynamic frantisek: checkbox at vm level to prevent accidental deletion shahar: vm payload issues (with cd-rom) Arik Changing the name of the virtual machines, created from a pool through web-admin-portal Alona: network as a main tab network permissions network main tab: search for networks (and api) network main tab: show vms by logical network hot-plug network - change networks on the fly Igor: hotplug nic hook Muli: display the VM interface IP per vnic report ipv6 addresses from guest vojtech: ui plugins Greg: cluster policy on hyperthreads usage http://www.ovirt.org/Features/cpu_thread_handling Clusters can now be configured to treat host cpu threads as cores for the purpose of vm resource allocation, migration, etc. Laszlo: -cpu host Ofri: quota - add to resource tab in power user portal moran: ovirt-live daniel erez: slm: move multiple disks (one by one) unicode enablement for OVF fields disks subtab/api under storage domain Tal: delete disk on vm delete flag (for image vs. volume) Liron: move OVF update out of reconstruct master and all other flows as an independent sync process Eli: inject events bootstrap: fetch logs to engine power management proxy (DC or cluster) multiple fencing devices add ilo2/ilo4 in text Yaniv Bronhaim: collect hardware (bios) information juan: engine-vdsm ssl session caching michael kublin: auto-recovery: last host in status up Alon Bar-Lev: bootstrap: install rhev-h without communication to engine 443 bootstrap: pki: use PKCS#12 format to store keys bootstrap: rewrite (ovirt-host-deploy) bootstrap: rewrite (otopi) Yair: rhevm-manage-domains hardcodes the DC's address instead of using DNS lookups Yaniv Dary: cloud provider report Report Daily list of people logged into Spice consoles and their activity add status of storage domain to dwh/reports DC Dashboard should display Storage Domains count single vm uptime - rhevm-reports should be able to report uptime against virtual machines Storage Inventory Report - New report michael pasternak: sdk - allow simultaneous connections to multiple servers cli - disable output redirection to ease testing efforts java sdk Doron: tuned profile for host doron/adam: not sure about status of vdsm-mom in 3.2? shireesh - what about: Forced removal of a host Configuration sync with Gluster CLI Import of existing gluster clusters Thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [oVirt Jenkins] ovirt_engine_dao_unit_tests - Build # 3091 - Still Failing!
On 01/20/2013 03:55 PM, Eyal Edri wrote: looks like this patch cause new unit test failure, can you check? http://gerrit.ovirt.org/gitweb?p=ovirt-engine.gita=commith=6fa50420e5c8365bea80125dc3e559d9edb57396 thanks eyal edri ovirt infra Thanks, indeed it does. Fix submitted: http://gerrit.ovirt.org/11236 - Original Message - From: Jenkins oVirt Server jenk...@ovirt.org To: ee...@redhat.com, engine-patc...@ovirt.org, ol...@redhat.com, yzasl...@redhat.com, in...@ovirt.org, gerr...@gerrit.ovirt.org Sent: Sunday, January 20, 2013 10:43:39 PM Subject: [oVirt Jenkins] ovirt_engine_dao_unit_tests - Build # 3091 - Still Failing! Project: http://jenkins.ovirt.org/job/ovirt_engine_dao_unit_tests/ Build: http://jenkins.ovirt.org/job/ovirt_engine_dao_unit_tests/3091/ Build Number: 3091 Build Status: Still Failing Triggered By: Started by upstream project ovirt_engine build number 4,062 - Changes Since Last Success: - Changes for Build #3084 [gerrit2] core, db: add cluster view [gerrit2] webadmin: show data center name for each cluster Changes for Build #3085 [asaf] restapi: A check for possible NPE BackendVmsResource:remove method Changes for Build #3086 [gerrit2] core: improve error msg text in storage connection Changes for Build #3087 Changes for Build #3088 Changes for Build #3089 [gerrit2] webadmin: move disks popup - fix alignment [gerrit2] core: prevent failure on concurrent live snapshots Changes for Build #3090 [gerrit2] core: required space validation - destination domain Changes for Build #3091 [gerrit2] engine: VmDeviceUtils cleanup [gerrit2] engine: VmDeviceUtils cleanup 2 - Failed Tests: - 1 tests failed. FAILED: org.ovirt.engine.core.dao.GeneralDbDAOTest.testBusinessEntitiesColumnSize Error Message: null Stack Trace: java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.ovirt.engine.core.dao.GeneralDbDAOTest.testBusinessEntitiesColumnSize(GeneralDbDAOTest.java:109) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174) at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104) at
Re: [Engine-devel] CPU Overcommit Feature
On 12/17/2012 05:52 PM, Andrew Cathrow wrote: ... and let's not call this CPU overcommit feature. It's nothing like that - it's Hyperthread handling - Original Message - From: Simon Grinberg si...@redhat.com To: Greg Padgett gpadg...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Monday, December 17, 2012 1:13:03 PM Subject: Re: [Engine-devel] CPU Overcommit Feature - Original Message - From: Greg Padgett gpadg...@redhat.com To: engine-devel engine-devel@ovirt.org Sent: Monday, December 17, 2012 4:37:57 PM Subject: [Engine-devel] CPU Overcommit Feature Hi, I've been working on a feature to allow CPU Overcommitment of hosts in a cluster. This first stage allows the engine to consider host cpu threads as cores for the purposes of VM resource allocation. This wiki page has further details, your comments are welcome! http://www.ovirt.org/Features/cpu_overcommit Basically looking good. Hyperthread though is vendor specific. For AMD it's Clustered Multi-Thread while for Intel it's Hyper-Thread Official name is simultaneous multithreading (SMT) but no one outside of the academy will recognize that. in libvirt if I read it right it's attribute name='thread_siblings' So why not just call it threads. We'll have cpuSockets, cpiCores, and cpuThreads, should be clear when in CPU context. In the GUI just change hyperthreads to CPU threads. While in the tool tip explain that it's either AMD Clustered Multi-Thread or Intel Hyperthread Thanks in advance, Greg ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Thanks Simon and Andrew. I've moved the wiki page [1] (with a redirect at the old name), updated the terms within to not be vendor-specific, and will do the same with the implementation. [1] http://www.ovirt.org/Features/cpu_thread_handling ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Getting the storage domain of a LunDisk
On 08/07/2012 10:06 AM, Hopper, Ricky wrote: Hi all, Does anyone know the best way to get the storage domain of a LunDisk object? I don't know if there's some query or anything I'm failing to find, but I can't find anything within the class itself that's definitive. Thanks, - Ricky ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel Hi Ricky, As far as I can see, the DiskImage class can give this--but AFAIK there is no way to convert a LunDisk to a DiskImage. I am curious if the storage domain info should be moved into the Disk class, which is a superclass of both LunDisk and DiskImage, or if perhaps there is a better place for it? Here's a partial class hierarchy that I mapped out when looking into this for a bug I'm working on: BaseDisk '- Disk |- LunDisk '- DiskImageBase '- DiskImage Thanks, Greg ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
