Re: [vdsm] How to figure out the transport type of Gluster volume from VDSM host (which is not a gluster peer) ?
2013-5-6 22:33, Deepak C Shetty: Hi Lists, I am looking at options to figure the transport type of a gluster volume (given the volfileserver:volname) from a host that is *not* part of gluster volume (aka not a gluster peer). The context here is GlusterFS as a storage domain in oVirt/VDSM, which is currently available in upstream oVirt. This features exploits the QEMU-GlusterFS native integration, where the VM disk is specified using gluster+://... protocol. For eg. if transport is TCP.. the URI looks liek gluster+tcp://..., otherwise gluster+rdma://... Thus, to generate the gluster QEMU URI in VDSM, i need to know the Gluster volume's transport type and the only inputs that oVirt gets for GlusterFS storage domain are... a) volfileserver (the host running glusterd) b) volname (the name of the volume) Currently i use VDSM's gluster plugin to do the eq. of "gluster volume info " to determine Gluster volume's transport type, but this won't work if the VDSM host is not a gluster peer, What do you mean by using "gluster peer"? Does "gluster peer" mean the host is running glusterd? which is a constraint! ... and I would like to fix/remove this constraint. So i discussed a bit on #glsuter-dev IRC and want to put down the options here for the community to help provide inputs on whats the best way to approach this... 1) Use gluster --remote-host= volume info This is not a supported way and there is no guarantee on how long the --remote-host option be supported in gluster, since it has some security issues 2) Use gluster ::system getspec I tried this but it never worked for me... whats the right way of using this ? For me.. it just returned back to shell w/o dumping the volfile at all! 3) Have oVirt user provide the transport type as well (while creating Gluster storgae domain) in addition to volfileserver:volname options This would be easiest, since VDSM can form the gluster QEMU URI by directly using the transport type specified by the user, and this won't have a need to use the vdsm-gluster plugin, hence no need for VDSM host to be part of gluster peer...but this would mean addnl input for user to provide during Gluster domain creation and oVirt UI changes to take the transport type as input in addition to volfileserver:volname What will happen if a user gives a wrong transport type to VDSM? Comments/Opinions/Inputs appreciated thanx, deepak (P.S. cross-posting this to VDSM and Gluster devel lists, as it relates to both) ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] vdsm sync call meeting minutes Monday April 8th 2013
Dan Kenigsberg: Speaking attendees, in no particular order: Ayal, Yaniv, Saggi, Douglas, Adam, Dan, Federico, Toni. Here are the meeting minutes for this week's sync call: - two small ovirt-3.2 patches are pending http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:ovirt-3.2,n,z Federico to consider to take them into the branch. - Federico to research more into Yuval M and Limor Gavish's nasty failure http://lists.ovirt.org/pipermail/users/2013-March/013635.html - Toni: No huge leaps for NetReload during this fortnight. - There are plenty of nice improvements by Zhou Zhen Sheng. Please help in reviewing them: http://gerrit.ovirt.org/#/dashboard/1000174 - Two Vdsm developers from IBM Beijing intend to come over to http://www.ovirt.org/OVirt_Global_Workshops#oVirt_Workshop_at_Intel_Campus So is /me. I post two topic proposals to workshop...@ovirt.org. No response? - Saggi is busy writing a Java client for Vdsm. Needs review for his jsonrpc patches. Will ping Adam about it. - It's high time to solve https://fedorahosted.org/ovirt/ticket/41 Adam is cool about Jenkis async task, nacking an invalid commit message. Dan prefers to get any sinner to follow this link https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=2MTZZR5UUAD5L&item_name=Secret%20Link%20Atonement&amount=2%2e00¤cy_code=USD But nagging ovirt infra about ticket 41 is acceptable. - Ayal: Backup API, DRBD design work is being done. - Subteams (mainly infra and storage) to update their list of features in http://www.ovirt.org/OVirt_3.3_release-management as the feature freaze is in less than 2 months. See you, Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] How to map the oVirt engine version to VDSM version by git tags?
Hi, I am looking for a way to map the oVirt version to VDSM version and engine version by git tags. I can run "git tag -l" under engine git workspace and vdsm git workspace. Here are the output from these two "git tag -l" command. Under oVirt engine workspace: -bash-4.1$ git tag -l ovirt-engine-3.0.0_0001 ovirt-engine-3.1.0 ovirt-engine-3.2.0 ovirt-engine-3.2.1 Under vdsm workspace: -bash-4.1$ git tag -l v4.10.0 v4.10.1 v4.10.2 v4.10.3 v4.9.0 v4.9.1 v4.9.2 v4.9.3 v4.9.3.1 v4.9.3.2 v4.9.3.3 v4.9.4 v4.9.5 v4.9.6 I can checkout the oVirt 3.2.1 snapshot in engine workspace by "git checkout ovirt-engine-3.2.1". But how can I get the VDSM snapshot by the tags in VDSM workspace? How can I know which change-set is for oVirt 3.2.1 in VDSM workspace? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Does we really need _addTask() in taskManager.py?
Hi, I am reading the code in taskManager.py. It seems that _addTask was defined in class Taskmanager, but no else to use them. Also, class Taskmanager get a similar function queue() which is used. Does we really need _addTask() function or just remove it? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] VDSM test unit running path
Hi, Now, The VDSM test unit is expected to run in a un-installed enviornment and use hackVDSM() to fake modules importing. hackVDSM() is pretty tricky for other modules to be added because of the dependency order. I would propose to run the VDSM unit tests in a installed VDSM path which is the same path as other VDSM runnable binaries. By this way, we can abandon hackVDSM() and run the unit test normally. On the other hand, the VDSM building script installs the VDSM files to /builder by default, it is reasonable to start the VDSM unit test after all parts of the building script finishes. Any comments about this idea? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] VDSM Repository Reorganization
there too. To be honest I started envisioning client/common/daemon/tool as pure-python directories (as they are once installed) so maybe the schema could be placed somewhere else with other miscellaneous files (./schema? ./config?), but I wouldn't stress over this if it's impractical. I would like to see also pure python directories. A rough quick n dirty overview how I could imagine it to look like: topdir/ build-aux/* contrib/* deployment/ bootstrap/ log/ reg/ doc/ vdsm api cli hooks/* m4/* tests/* scripts/ vdsm/* storage/* network/* vdsm/ api/ json/ jsonrpc/ BindingJsonRpc.py xml/ BindingXMLRPC.py Bridge.py vdsmapi-schema.json process-schema.py vdsmapi.py cli/ vdsClient.py VdsClientGluster.py extra/ betterPopen/ shared/ vdsm/ SecureXMLRPCServer.py config.py.in constants.py.in utils.py vdscli.py.in* utils.py tool/ daemon/ storage/* network/* vdsmd/ vdsm * I would like to make all of the service under vdsm/daemon directory only reference the files under vdsm/shared. And back-reference from shared/vdsm to vdsm/daemon is not allowed. In current VDSM repository, the files under vdsm/storage and the files under vdsm are cross-referenced on each other, that is a disaster to the effort trying to make stand-alone service from storage service outside of vdsmd. In addition, the files under vdsm/cli should also only depend on the files under vdsm/shared. BTW: Another service under vdsm/daemon should be supervdsm, so the daemon should like: ... daemon/ storage/* network/* vdsmd/ supervdsmd/ ... That said, I need to assert that I am not yet fully aware of all connections between the files and I guess the maintainers will know best about their respective parts of responsibility. I'd like to add that it'd be probably a good idea to start having network and storage split off the main daemon for maintenance reasons, even if they interface each other. I personally even could see a scenario where network and storage are working as standalone daemons. However that's off topic for know I guess. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Meeting minutes, Jan 28
于 2013-1-29 4:00, Dan Kenigsberg 写道: Following are the notes that I've taking during the call. I'm pretty sure that I'm missing something important, such as another blocker BZ, so please reply with corrections/addtions. - Adam: going to introduce Python binding for the new Vdsm API - Sharad has joined our call for the first time. Welcome! Interested in backup/restore functionality in oVirt, to facilitate integration with TSM, presumably http://www-142.ibm.com/software/products/us/en/tivostormana/ - Federico warns that live snapshot is not enough, since we lack live merge. After a year of daily backups, each qcow chain would be 365 hops deep. That's horrible. It looks like that Qemu 1.3 and libvirt 1.0.0 already supports the "live block commit" which is the base of live commit. Do we have plan to support "live commit" in next oVirt release? ovirt-3.2: - We should probably revert http://gerrit.ovirt.org/9315 (integrate zombie reaper in supervdsmServer) as it tickles a bug in multiprocessing. See http://lists.ovirt.org/pipermail/users/2013-January/011857.html for more details in the thread starting in http://lists.ovirt.org/pipermail/users/2013-January/011747.html (thanks for raising the issue and finding the culprit, Dead Horse) - Danken believes that all important el6-specific issues reported by dreyou are now handled in the master and ovirt-3.2 branches. Please report this list if he's wrong. - Federico reports that udev is going to revert to its Fedora 18alpha behavior so that we can keep our integration with it. I'd love to see the udev BZ listed in our tracker https://bugzilla.redhat.com/showdependencytree.cgi?id=881006&hide_resolved=1 as I do not recall it. Federico, could you add it? We should probably require a newer udev release. - Federico reports that we must take a short-but-important patch from Lee Yarwood: http://gerrit.ovirt.org/#/c/11281/ Cross-distro: - Adam: a guy from IBM is now listing issues that block running Vdsm on Ubuntu. Yay! - Saggi suggests that IBM contribute an Ubuntu slave for Jenkins, so that each and every patch is tested not to introduce Ubuntu regressions - just as Red Hat has recently done with EL6. - Danken: yes, we have got el6-based unit-testing running. If you are still getting an unjustified X, you may need to rebase on current master. - Federico: NetworkManager 9 has bridge support, which leads the way to NM-based implementation of Vdsm's configNetwork. - Danken: Toni is working on a suggestion for refactoring configNetwork in such a way that multiple implementations (e.g. ifcfg-based, NM-based) can coexist. Stay stuned for his Wiki page on the subject. Other refactoring going on: - Vinzenz has begun breaking the horrible libvirtvm+vm to edible-size pieces. Reviews are most welcome, particularly from Mark Wu, whose http://gerrit.ovirt.org/10054/ only awaits verification tick. Functional Testing: - Adam: a guy from IBM (Zhou, I presume) is running the functional tests via Jenkins on his laptop. Yay! Zhou Zheng Sheng - Adam: plans to add a functional test for the getVdsCaps verb, as he found out that an evil Vdsm developer has added values to the caps without updating the schema. - P.S. FlowID is now in for storage verbs as well (thanks, Douglas). I hope Haim is happier now. Happy coding! ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: New Storage API
2013-1-15 5:34, Ayal Baron: image and volume are overused everywhere and it would be extremely confusing to have multiple meanings to the same terms in the same system (we have image today which means virtual disk and volume which means a part of a virtual disk). Personally I don't like the distinction between image and volume done in ec2/openstack/etc seeing as they're treated as different types of entities there while the only real difference is mutability (images are read-only, volumes are read-write). To move to the industry terminology we would need to first change all references we have today to image and volume in the system (I would say also in ovirt-engine side) to align with the new meaning. Despite my personal dislike of the terms, I definitely see the value in converging on the same terminology as the rest of the industry but to do so would be an arduous task which is out of scope of this discussion imo (patches welcome though ;) Another distinction between Openstack and oVirt is how the Nova/ovirt-engine look upon storage systems. In Openstack, a stand alone storage service(Cinder) exports the raw storage block device to Nova. On the other hand, in oVirt, storage system is highly bounded with the cluster scheduling system which integrates storage sub-system, VM dispatching sub-system, ISO image sub systems. This combination make all of the sub-system integrated in a whole which is easy to deploy, but it make the sub-system more opaque and not harder to reuse and maintain. This new storage API proposal give us an opportunity to distinct these sub-systems as new components which export better, loose-coupling APIs to VDSM. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live storage migration status in oVirt 3.2
于 2013-1-11 19:07, Daniel Erez 写道: - Original Message - From: "Shu Ming" To: engine-de...@ovirt.org, "VDSM Project Development" Cc: "Federico Simoncelli" , de...@redhat.com Sent: Friday, January 11, 2013 2:45:47 AM Subject: Live storage migration status in oVirt 3.2 Hi, I am reviewing the live storage migration with my oVirt environment updated from the public nightly repository two weeks ago. I found that the "move" button was still gray as before when the VM was up. Only after I deactivated the disk, did the button become into non-gray state. I am wondering if the live storage migration will be supported in oVirt 3.2 release or not. BTW: These patches below should enable the live storage migration already, but I can not see it enabled in my engine. http://gerrit.ovirt.org/5252 Change I91e641cb: pool: live storage migration implementation http://gerrit.ovirt.org/8105 Change subject: core: Live Storage Migration commands http://gerrit.ovirt.org/8103 Change subject: core: VDS Commands for Live Storage Migration http://gerrit.ovirt.org/8102 Change subject: core: Adding VDSM API for Live Storage Migration http://gerrit.ovirt.org/8470 Change subject: webadmin: Adding Live Storage Migration support http://gerrit.ovirt.org/8857 core: disable LiveStorageMigration on 3.1 -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC Hi Shu, When a VM is up, the "move" button is enabled only for Data Center version 3.2. Can you please verify that the selected disk resides on a 3.2 Data Center? I checked my data center compatibility version is 3.1. However, it is interesting that the compatibility version of the cluster in the data center is 3.2 Does the data center allow a higher version cluster? I will try to upgrade the data center version and try the migration again. Best Regards, Daniel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Live storage migration status in oVirt 3.2
Hi, I am reviewing the live storage migration with my oVirt environment updated from the public nightly repository two weeks ago. I found that the "move" button was still gray as before when the VM was up. Only after I deactivated the disk, did the button become into non-gray state. I am wondering if the live storage migration will be supported in oVirt 3.2 release or not. BTW: These patches below should enable the live storage migration already, but I can not see it enabled in my engine. http://gerrit.ovirt.org/5252 Change I91e641cb: pool: live storage migration implementation http://gerrit.ovirt.org/8105 Change subject: core: Live Storage Migration commands http://gerrit.ovirt.org/8103 Change subject: core: VDS Commands for Live Storage Migration http://gerrit.ovirt.org/8102 Change subject: core: Adding VDSM API for Live Storage Migration http://gerrit.ovirt.org/8470 Change subject: webadmin: Adding Live Storage Migration support http://gerrit.ovirt.org/8857 core: disable LiveStorageMigration on 3.1 -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] ovirt nightly FC17 public repositories broken?
Hi, Any one was encountering this error message when using yum on ovirt public repositories? http://ovirt.org/releases/nightly/rpm/Fedora/17/repodata/primary.xml.gz <http://ovirt.org/releases/nightly/rpm/Fedora/17/repodata/primary.xml.gz>: [Errno -1] Metadata file does not match checksum My vdsm host and engine server was installed with FC17+virt-review, so I used FC17 nightly releases to update my VDSM and engine packages. After a bit investigation, I found these issues. [root@node1-sming ovirt-nightly]# ls -lh * -rw-r--r--. 1 root root 0 Jan 7 16:47 cachecookie -rw-r--r--. 1 root root 702K Jan 7 14:22 filelists.xml.gz -rw-r--r--. 1 root root 132K Jan 7 14:22 other.xml.gz -rw-r--r--. 1 root root 279K Jan 7 14:22 primary.xml.gz -rw-r--r--. 1 root root 1.4K Jan 5 14:35 repomd.xml *<---the date was always Jan 5 different from other files even http://resources.ovirt.org/releases/nightly**/rpm/Fedora/17/repodata/ showed it was Jan 7, also I tried to use wget to download this file and got the same thing.** * [root@node1-sming ovirt-nightly]# sha1sum other.xml.gz filelists.xml.gz primary.xml.gz 453559e86950af07c876931f318d1e92ab58f289 other.xml.gz 5318e237c0f4d2ea3d1ec24e82b9f9bbe9d23a31 filelists.xml.gz e667267d45b576844a5f2cb27ce7bcfb8bb9b4f0 primary.xml.gz [root@node1-sming ovirt-nightly]# cat repomd.xml |grep open-checksum ce6a5243ee81c124d0fe48d103174b93fabe60573e80b5851888298373d3be9e 554a301c526d335ca4a7985e7fc7b0374bca8c061ff5addbd164de81996dc5a0 be3f9e6860cd9a4ac4f3f5000ad503175e7ac79a43185a778d4ab732c0d8190c *Note: These checksum from repmod.xml were quite different from the result of sha1sum above.* Any clues here? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] FW: Querying for and registering unknown disk images on a storage domain
2012-12-20 23:18, Morrissey, Christopher: Hi All, I've been working on a bit of functionality for the engine that will allow a user to query a domain for new disk images (GetUnregisteredImagesQuery) for which the engine was previously unaware and a separate command to register those images (ImportImageCommand). These commands will be exposed through the REST API. This functionality is needed as we are developing an extension/plugin to oVirt that will allow a NetApp storage controller to handle cloning the actual disks outside of oVirt and need to import them once they are cloned. We'll be using other existing APIs to attach the disk to the necessary VM once the disk is cloned. On the NetApp side, we'll ensure the disk is coalesced before cloning so as to avoid the issues of registering snapshots. I am just curious about how the third party tool like NetApp to make sure the disk of a running VM coalesced before cloning? By an agent in the VM to flush file-system cache out to the disk? GetUnregisteredImagesQuery will be accessible through the disks resource collection on a storage domain. A "disks" resource collection does not yet exist and will need to be added. To access the unregistered images, a parameter (maybe "unregistered=true") would be passed. So the path to "GET" the unregistered disk images on a domain would be something like /api/storagedomains/f0dbcb33-69d3-4899-9352-8e8a02f01bbd/disks?unregistered=true. This will return a list of disk images that can be each used as input to the ImportImageCommand to get them added to oVirt. ImportImageCommand will be accessible through "POST"ing a disk to /api/disks?import=true. The disk will be added to the oVirt DB based on the information supplied and afterward would be available to attach to a VM. When querying for unregistered disk images, the GetUnregisteredImagesQuery command will use the getImagesList() VDSM command. Currently this only reports the GUIDs of all disk images in a domain. I had been using the getVolumesList() and getVolumeInfo() VDSM commands to fill in the information so that valid disk image objects could be registered in oVirt. It seems these two functions are set to be removed since they are too invasive into the internal VDSM workings. The VDSM team will need to either return more information about each disk as part of the getImagesList() function or add a new function getImageInfo() that will give the same information for a given image GUID. Here is the project proposal for floating disk in oVirt. I think unregistered images are also floating disks. http://www.ovirt.org/Features/DetailedFloatingDisk Note that much of this work had originally been submitted under patch http://gerrit.ovirt.org/#/c/9603/. After several reviews it was found to be lacking in its design and was using deprecated APIs that did not yet have replacements. I'm reworking the code now to conform to this design and asking for further input from the VDSM, core, and restapi teams to ensure we can get this done quickly and correctly as it is needed for the 3.2 release. -Chris *Chris Morrissey* Software Engineer NetApp Inc. 919.476.4428 ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- ?? Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Host bios information
After a quick review of the wiki page, it was stated that dmidecode gave too much informations. Only five fields will be displayed in the hardware tab, "Manufactory", "Version", "Family", "UUID" and "serial number". For "Family", it is mean the CPU core's family. And it confuses me a bit with the "CPU name" and "CPU type" fields in general tab. I think we should chose the best one to characterizethe CPU type. ybronhei: Today in the Api we display general information about the host that vdsm export by getCapabilities Api. We decided to add bios information as part of the information that is displayed in UI under host's general sub-tab. To summaries the feature - We'll modify General tab to Software Information and add another tab for Hardware Information which will include all the bios data that we'll decide to gather from the host and display. Following this feature page: http://www.ovirt.org/Features/Design/HostBiosInfo for more details. All the parameters that can be displayed are mentioned in the wiki. I would greatly appreciate your comments and questions. Thanks. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] RFC: New Storage API
2012-12-11 4:36, Saggi Mizrahi: - Original Message - From: "Adam Litke" To: "Saggi Mizrahi" Cc: "Shu Ming" , "engine-devel" , "VDSM Project Development" Sent: Monday, December 10, 2012 1:39:51 PM Subject: Re: [vdsm] RFC: New Storage API On Thu, Dec 06, 2012 at 11:52:01AM -0500, Saggi Mizrahi wrote: - Original Message - From: "Shu Ming" To: "Saggi Mizrahi" Cc: "VDSM Project Development" , "engine-devel" Sent: Thursday, December 6, 2012 11:02:02 AM Subject: Re: [vdsm] RFC: New Storage API Saggi, Thanks for sharing your thought and I get some comments below. Saggi Mizrahi: I've been throwing a lot of bits out about the new storage API and I think it's time to talk a bit. I will purposefully try and keep implementation details away and concentrate about how the API looks and how you use it. First major change is in terminology, there is no long a storage domain but a storage repository. This change is done because so many things are already called domain in the system and this will make things less confusing for new-commers with a libvirt background. One other changes is that repositories no longer have a UUID. The UUID was only used in the pool members manifest and is no longer needed. connectStorageRepository(repoId, repoFormat, connectionParameters={}): repoId - is a transient name that will be used to refer to the connected domain, it is not persisted and doesn't have to be the same across the cluster. repoFormat - Similar to what used to be type (eg. localfs-1.0, nfs-3.4, clvm-1.2). connectionParameters - This is format specific and will used to tell VDSM how to connect to the repo. Where does repoID come from? I think repoID doesn't exist before connectStorageRepository() return. Isn't repoID a return value of connectStorageRepository()? No, repoIDs are no longer part of the domain, they are just a transient handle. The user can put whatever it wants there as long as it isn't already taken by another currently connected domain. disconnectStorageRepository(self, repoId) In the new API there are only images, some images are mutable and some are not. mutable images are also called VirtualDisks immutable images are also called Snapshots There are no explicit templates, you can create as many images as you want from any snapshot. There are 4 major image operations: createVirtualDisk(targetRepoId, size, baseSnapshotId=None, userData={}, options={}): targetRepoId - ID of a connected repo where the disk will be created size - The size of the image you wish to create baseSnapshotId - the ID of the snapshot you want the base the new virtual disk on userData - optional data that will be attached to the new VD, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new VD I think we will also need a function to check if a a VirtualDisk is based on a specific snapshot. Like: isSnapshotOf(virtualDiskId, baseSnapshotID): No, the design is that volume dependencies are an implementation detail. There is no reason for you to know that an image is physically a snapshot of another. Logical snapshots, template information, and any other information can be set by the user by using the userData field available for every image. Statements like this make me start to worry about your userData concept. It's a sign of a bad API if the user needs to invent a custom metadata scheme for itself. This reminds me of the abomination that is the 'custom' property in the vm definition today. In one sentence: If VDSM doesn't care about it, VDSM doesn't manage it. userData being a "void*" is quite common and I don't understand why you would thing it's a sign of a bad API. Further more, giving the user choice about how to represent it's own metadata and what fields it want to keep seems reasonable to me. Especially given the fact that VDSM never reads it. The reason we are pulling away from the current system of VDSM understanding the extra data is that it makes that data tied to VDSMs on disk format. VDSM on disk format has to be very stable because of clusters with multiple VDSM versions. Further more, since this is actually manager data it has to be tied to the manager backward compatibility lifetime as well. Having it be opaque to VDSM ties it to only one, simpler, support lifetime instead of two. Making userData being opaque gives flexibilities to the management applications. To me, opaque userDaa can have two types at least. The first is the userData for runtime only. The second is the userData expected to be persisted into the metadata disk. For the first type, the management applications can store their own data structures like temporary task states, VDSM query caches &etc. After the VDSM host is fenced, th
Re: [vdsm] RFC: New Storage API
于 2012-12-7 13:23, Deepak C Shetty: On 12/06/2012 10:22 PM, Saggi Mizrahi wrote: - Original Message - From: "Shu Ming" To: "Saggi Mizrahi" Cc: "VDSM Project Development" , "engine-devel" Sent: Thursday, December 6, 2012 11:02:02 AM Subject: Re: [vdsm] RFC: New Storage API Saggi, Thanks for sharing your thought and I get some comments below. Saggi Mizrahi: I've been throwing a lot of bits out about the new storage API and I think it's time to talk a bit. I will purposefully try and keep implementation details away and concentrate about how the API looks and how you use it. First major change is in terminology, there is no long a storage domain but a storage repository. This change is done because so many things are already called domain in the system and this will make things less confusing for new-commers with a libvirt background. One other changes is that repositories no longer have a UUID. The UUID was only used in the pool members manifest and is no longer needed. connectStorageRepository(repoId, repoFormat, connectionParameters={}): repoId - is a transient name that will be used to refer to the connected domain, it is not persisted and doesn't have to be the same across the cluster. repoFormat - Similar to what used to be type (eg. localfs-1.0, nfs-3.4, clvm-1.2). connectionParameters - This is format specific and will used to tell VDSM how to connect to the repo. Where does repoID come from? I think repoID doesn't exist before connectStorageRepository() return. Isn't repoID a return value of connectStorageRepository()? No, repoIDs are no longer part of the domain, they are just a transient handle. The user can put whatever it wants there as long as it isn't already taken by another currently connected domain. So what happens when user mistakenly gives a repoID that is in use before.. there should be something in the return value that specifies the error and/or reason for error so that user can try with a new/diff repoID ? I think let the user to give the repoID is meaningless and error-prune. Developer must maintain a a unique ID list for every storage repository connected. disconnectStorageRepository(self, repoId) In the new API there are only images, some images are mutable and some are not. mutable images are also called VirtualDisks immutable images are also called Snapshots There are no explicit templates, you can create as many images as you want from any snapshot. There are 4 major image operations: createVirtualDisk(targetRepoId, size, baseSnapshotId=None, userData={}, options={}): targetRepoId - ID of a connected repo where the disk will be created size - The size of the image you wish to create baseSnapshotId - the ID of the snapshot you want the base the new virtual disk on userData - optional data that will be attached to the new VD, could be anything that the user desires. options - options to modify VDSMs default behavior IIUC, i can use options to do storage offloads ? For eg. I can create a LUN that represents this VD on my storage array based on the 'options' parameter ? Is this the intended way to use 'options' ? returns the id of the new VD I think we will also need a function to check if a a VirtualDisk is based on a specific snapshot. Like: isSnapshotOf(virtualDiskId, baseSnapshotID): No, the design is that volume dependencies are an implementation detail. There is no reason for you to know that an image is physically a snapshot of another. Logical snapshots, template information, and any other information can be set by the user by using the userData field available for every image. createSnapshot(targetRepoId, baseVirtualDiskId, userData={}, options={}): targetRepoId - The ID of a connected repo where the new sanpshot will be created and the original image exists as well. size - The size of the image you wish to create baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want to snapshot userData - optional data that will be attached to the new Snapshot, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new Snapshot copyImage(targetRepoId, imageId, baseImageId=None, userData={}, options={}) targetRepoId - The ID of a connected repo where the new image will be created imageId - The image you wish to copy baseImageId - if specified, the new image will contain only the diff between image and Id. If None the new image will contain all the bits of image Id. This can be used to copy partial parts of images for export. userData - optional data that will be attached to the new image, could be anything that the user desires. options - options to modify VDSMs default behavior Does this function mean that we can copy the image from one repository to another repository? Do
Re: [vdsm] RFC: New Storage API
tatus of the image. The status of the image can be either optimized, degraded, or broken. "Optimized" means that the image is available and you can run VMs of it. "Degraded" means that the image is available and will run VMs but it might be a better way VDSM can represent the underlying data. What does the "represent" mean here? "Broken" means that the image can't be used at the moment, probably because not all the data has been set up on the volume. Apart from that VDSM will also return the last persisted status information which will conatin hostID - the last host to try and optimize of fix the image Any host can optimize the image? No need to be SDM? stage - X/Y (eg. 1/10) the last persisted stage of the fix. percent_complete - -1 or 0-100, the last persisted completion percentage of the aforementioned stage. -1 means that no progress is available for that operation. last_error - This will only be filled if the operation failed because of something other then IO or a VDSM crash for obvious reasons. It will usually be set if the task was manually stopped The user can either be satisfied with that information or as the host specified in host ID if it is still working on that image by checking it's running tasks. So we need a function to know what tasks are running on the image checkStorageRepository(self, repositoryId, options={}): A method to go over a storage repository and scan for any existing problems. This includes degraded\broken images and deleted images that have no yet been physically deleted\merged. It returns a list of Fix objects. Fix objects come in 4 types: clean - cleans data, run them to get more space. optimize - run them to optimize a degraded image merge - Merges two images together. Doing this sometimes makes more images ready optimizing or cleaning. The reason it is different from optimize is that unmerged images are considered optimized. mend - mends a broken image The user can read these types and prioritize fixes. Fixes also contain opaque FIX data and they should be sent as received to fixStorageRepository(self, repositoryId, fix, options={}): That will start a fix operation. All major operations automatically start the appropriate "Fix" to bring the created object to an optimize\degraded state (the one that is quicker) unless one of the options is AutoFix=False. This is only useful for repos that might not be able to create volumes on all hosts (SDM) but would like to have the actual IO distributed in the cluster. Other common options is the strategy option: It has currently 2 possible values space and performance - In case VDSM has 2 ways of completing the same operation it will tell it to value one over the other. For example, whether to copy all the data or just create a qcow based of a snapshot. The default is space. You might have also noticed that it is never explicitly specified where to look for existing images. This is done purposefully, VDSM will always look in all connected repositories for existing objects. For very large setups this might be problematic. To mitigate the problem you have these options: participatingRepositories=[repoId, ...] which tell VDSM to narrow the search to just these repositories and imageHints={imgId: repoId} which will force VDSM to look for those image ID just in those repositories and fail if it doesn't find them there. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] moving the collection of statistics to external process
于 2012-12-6 4:51, Itamar Heim 写道: On 12/05/2012 10:33 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 10:21:39PM +0200, Itamar Heim wrote: On 12/05/2012 10:16 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 09:01:24PM +0200, Itamar Heim wrote: On 12/05/2012 08:57 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 08:30:10PM +0200, Itamar Heim wrote: On 12/05/2012 04:42 PM, Adam Litke wrote: I wanted to know what do you think about it and if you have better solution to avoid initiate so many threads? And if splitting vdsm is a good idea here? In first look, my opinion is that it can help and would be nice to have vmStatisticService that runs and writes to separate log the vms status. Vdsm recently started requiring the MOM package. MOM also performs some host and guest statistics collection as part of the policy framework. I think it would be a really good idea to consolidate all stats collection into MOM. Then, all stats become usable within the policy and by vdsm for its own internal purposes. Today, MOM has one stats collection thread per VM and one thread for the host stats. It has an API for gathering the most recently collected stats which vdsm can use. isn't this what collectd (and its libvirt plugin) or pcp are already doing? Lot's of things collect statistics, but as of right now, we're using MOM and we're not yet using collectd on the host, right? I think we should have a single stats collection service and clients for it. I think mom and vdsm should get their stats from that service, rather than have either beholden to any new stats something needs to collect. How would this work for collecting guest statistics? Would we require collectd to be installed in all guests running under oVirt? my understanding is collectd is installed on the host, and uses collects libvirt plugin to collect guests statistics? Yes, but some statistics can only be collected by making a call to the oVirt guest agent (eg. guest memory statistics). The logical next step would be to write a collectd plugin for ovirt-guest-agent, but vdsm owns the connections to the guest agents and probably does not want to multiplex those connections for many reasons (security being the main one). and some will come from qemu-ga which libvirt will support? maybe a collectd vdsm plugin for the guest agent stats? I am thinking to have the collectd as a stand alone service to collect the statics from both ovirt-guest and qemu-ga. Then collected can export the information to host proc file system in layered architecture. Then mom or other vdsm service can get the information from the proc file system like other OS statics exported in the host. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [VDSM][RFC] hsm service standalone
2012-12-5 23:55, ybronhei: On 12/03/2012 07:22 PM, Saggi Mizrahi wrote: HSM is not a package it's an application. Currently it and the rest of VDSM share the same process but they use RPC to communicate. This is done so that one day we can actually have them run as different processes. HSM is not something you import, it's a daemon you communicate with. - Original Message - From: "Dan Kenigsberg" To: "Saggi Mizrahi" Cc: "Sheldon" , a...@linux.vnet.ibm.com, vdsm-devel@lists.fedorahosted.org, "Zheng Sheng ZS Zhou" Sent: Monday, December 3, 2012 12:01:28 PM Subject: Re: [vdsm] [VDSM][RFC] hsm service standalone On Mon, Dec 03, 2012 at 11:35:44AM -0500, Saggi Mizrahi wrote: There are a bunch of precondition to having HSM pulled out. On simple things is that someone needs to go through storage/misc.py and utils.py and move all the code out to logical packages. There also needs to be a bit of a rearrangement of the code files so they can import the reusable code properly. I am also still very much against putting core VDSM in to site-packages. Would you elaborate on your position? Do you mind the wrong implications this may give about API stability? Contrary, It may help to split internal api and external (means rpc between vdsm to engine and rpc between vdsm and hsm), and it might be easier to understand the process flow and control the request from engine. For example, for migration you will receive vdsm request for migration, and that will initiate all the internal processes that we do during migration by sending requests to hsm service. The engine doesn't supposed to care about the internal rpc and those abilities does not supposed to be exposed via vdsm api if we don't allow those specific hsm operation from engine, but via HSM-python service that won't be available for the engine. Might be a good division. But lots of work indeed.. Fortunately, the hsm directory in vdsm is almost self-contained. hsm files are depending on some files like config.py, constants.py &etc which are already in a vdsm site packages. After HSM-python package is created, both vdsm-4.1x and HSM-python-4.1.x package will depend on vdsm-python-4.1x package. vdsm-python-4.1x package might be a confusing name that time. It might be better named "vdsm-common-python-4.1x". ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [VDSM][RFC] hsm service standalone
Saggi, It might be that the wiki page was a bit confusing. Let me articulate it again and help others to understand the intention of this project. 1) Now, HSM files here refers to the files under "vdsm/storage" which are storage service related code. And other VDSM files call into those functions in HSM files in local way(different from RPC). So in the production environment, HSM files are linked in the VDSM process and there is no additional HSM process standing in the ovirt node. 2) This project goal is to move all the HSM files into self-contained workspace and a new stand alone HSM process will be started from these HSM files. Also a new HSM package will be built from this new self-contained workspace. Also all of the HSM APIs are public to the original VDSM service by RPC calls. It is possible that the HSM standalone workspace and the original VDSM workspace may share some functions in misc.py,task.py&etc. We can move those common files to vdsm site packages as the other common files. After this goal is achieved, we have one more HSM process in production environment and one more release package for HSM files. For backward compatibility, vdsm will run in two possible modes. VDSM service will try to discover if HSM RPC APIs are available from the new HSM process. If the discovery is successful, VDSM service will make RPC calls to the HSM process. If the discovery fails, it will fall back the legacy way calling the HSM functions by local way. 3) In order to achieve this goal, we should remove the preconditions gradually and safely. The first step here is the patch http://gerrit.ovirt.org/#/c/9345/ That try to decouple the HSM configuration parameters from the vdsm.conf and store them into a new different conf file. Saggi Mizrahi: HSM is not a package it's an application. Currently it and the rest of VDSM share the same process but they use RPC to communicate. This is done so that one day we can actually have them run as different processes. HSM is not something you import, it's a daemon you communicate with. - Original Message - From: "Dan Kenigsberg" To: "Saggi Mizrahi" Cc: "Sheldon" , a...@linux.vnet.ibm.com, vdsm-devel@lists.fedorahosted.org, "Zheng Sheng ZS Zhou" Sent: Monday, December 3, 2012 12:01:28 PM Subject: Re: [vdsm] [VDSM][RFC] hsm service standalone On Mon, Dec 03, 2012 at 11:35:44AM -0500, Saggi Mizrahi wrote: There are a bunch of precondition to having HSM pulled out. On simple things is that someone needs to go through storage/misc.py and utils.py and move all the code out to logical packages. There also needs to be a bit of a rearrangement of the code files so they can import the reusable code properly. I am also still very much against putting core VDSM in to site-packages. Would you elaborate on your position? Do you mind the wrong implications this may give about API stability? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [VDSM][RFC] hsm service standalone
Added Saggi, Ayal, Federico into this loop. And got two comments below. Sheldon: there is already a wiki page about hsm service standalone. http://wiki.ovirt.org/HSM_service_stand_alone And now I'm trying to split hsm as a standalone rpm according to the wiki proposal. as the wiki describe: V)The configuration parameters in vdsm.conf should be pull out to another hsm.conf file. And all VDSM process should not read hsm.conf directly. If it is necessary to get the HSM configurations, an API will be added into VDSM process for hsm configuration querying I have split the the vdsm/config.py and submit a patch http://gerrit.ovirt.org/#/c/9345/ I moved 'irs'section from vdsm/config.py to a new vdsm/storage/config.py for hsm But some of vdsm python module need to get configure items form 'irs'section, such as vdsm/clientIF.py, vdsm/libvirtvm.py. I moved vdsm/storage/config.py to site-packages/hsm when install hsm. And I import all the items form 'irs'section, by "from hsm import config as hsmconfig" Is it a good way to split the config.py? Also I have split the the constants.py and submit a patch http://gerrit.ovirt.org/#/c/9348/ And should I create a new "hsm" user for the DISKIMAGE_USER instead of "vdsm" user? as the wiki describe: VII) All of the python modules shared by both VDSM service and HSM service should go into python site-packages directory which is like /usr/lib/python2.7/site-packages/vdsm Now, task.py doesn't sit in this directory, we should fix it in this proposal. Task model is only for storage service now. We have plan to make task models to all VDSM service. So it is reasonable to move task.py into site-packages directory. about the site-packages directory, there are three proposals. 1. VDSM service and HSM service share the same site-packages directory, still used site-packages/vdsm and that means there will be still a vdsm-python rpm, and both vdsm and hsm depends on them. 2. VDSM service and HSM service share the same site-packages directory, but it will renamed as used site-packages/hsm Now most codes of hsm and vdsm import modules from site-packages vdsm, so all of they should be fixed. 3. VDSM service depend on vdsm-python rpm, and HSM service depend on hsm-python rpm. I need to re-sort the public modules to hsm-python or vdsm-python rpm. such as fileUtils.py, misc.py, BetterPopen and qemuImg.py which is better of the above three? Packaging HSM site package files into a another new package other than vdsm-python package doesn't prevent those files contained into a same targeting package. So at least we can do, two different package say, vdsm-core-python, vdsm-storage-python and the files in them are targeting the same directory /usr/lib/python2.7/site-packages/vdsm. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Future of Vdsm network configuration - Thread mid-summary
Adam Litke: On Mon, Nov 26, 2012 at 02:57:19PM +0200, Livnat Peer wrote: On 26/11/12 03:15, Shu Ming wrote: Livnat, Thanks for your summary. I got comments below. 2012-11-25 18:53, Livnat Peer: Hi All, We have been discussing $subject for a while and I'd like to summarized what we agreed and disagreed on thus far. The way I see it there are two related discussions: 1. Getting VDSM networking stack to be distribution agnostic. - We are all in agreement that VDSM API should be generic enough to incorporate multiple implementation. (discussed on this thread: Alon's suggestion, Mark's patch for adding support for netcf etc.) - We would like to maintain at least one implementation as the working/up-to-date implementation for our users, this implementation should be distribution agnostic (as we all acknowledge this is an important goal for VDSM). I also think that with the agreement of this community we can choose to change our focus, from time to time, from one implementation to another as we see fit (today it can be OVS+netcf and in a few months we'll use the quantum based implementation if we agree it is better) 2. The second discussion is about persisting the network configuration on the host vs. dynamically retrieving it from a centralized location like the engine. Danken raised a concern that even if going with the dynamic approach the host should persist the management network configuration. About dynamical retrieving from a centralized location, when will the retrieving start? Just in the very early stage of host booting before network functions? Or after the host startup and in the normal running state of the host? Before retrieving the configuration, how does the host network connecting to the engine? I think we need a basic well known network between hosts and the engine first. Then after the retrieving, hosts should reconfigure the network for later management. However, the timing to retrieve and reconfigure are challenging. We did not discuss the dynamic approach in details on the list so far and I think this is a good opportunity to start this discussion... From what was discussed previously I can say that the need for a well known network was raised by danken, it was referred to as the management network, this network would be used for pulling the full host network configuration from the centralized location, at this point the engine. About the timing for retrieving the configuration, there are several approaches. One of them was described by Alon, and I think he'll join this discussion and maybe put it in his own words, but the idea was to 'keep' the network synchronized at all times. When the host have communication channel to the engine and the engine detects there is a mismatch in the host configuration, the engine initiates 'apply network configuration' action on the host. Using this approach we'll have a single path of code to maintain and that would reduce code complexity and bugs - That's quoting Alon Bar Lev (Alon I hope I did not twisted your words/idea). On the other hand the above approach makes local tweaks on the host (done manually by the administrator) much harder. I worry a lot about the above if we take the dynamic approach. It seems we'd need to introduce before/after 'apply network configuration' hooks where the admin could add custom config commands that aren't yet modeled by engine. Any other approaches ? Static configuration has the advantage of allowing a host to bring itself back online independent of the engine. This is also useful for anyone who may want to deploy a vdsm node in standalone mode. I think it would be possible to easily support a quasi-static configuration mode simply by extending the design of the dynamic approach slightly. In dynamic mode, the network configuration is passed down as a well-defined data structure. When a particular configuration has been committed, vdsm could write a copy of that configuration data structure to /var/run/vdsm/network-config.json. During a subsequent boot, if the engine cannot be contacted after activating the management network, the cached configuration can be applied using the same code as for dynamic mode. We'd have to flesh out the circumstances under which this would happen. Multiple physical network devices are quite popular in x86 servers now. Can we add some smart algorithm to leverage multiple physical network devices? Say, one specific device for the management network and others for vdsm host networks. By isolating the management and host networks, the vdsm host can maintain a permanent management work and it is much more clean and not bug prone. If the host doesn't get multiple network devices, we should fall back to the traditional way. I'd like to add a more general question to the discussion what are the advantages of taking the dynamic approach? So far I collected tw
Re: [vdsm] Future of Vdsm network configuration - Thread mid-summary
Livnat, Thanks for your summary. I got comments below. 2012-11-25 18:53, Livnat Peer: Hi All, We have been discussing $subject for a while and I'd like to summarized what we agreed and disagreed on thus far. The way I see it there are two related discussions: 1. Getting VDSM networking stack to be distribution agnostic. - We are all in agreement that VDSM API should be generic enough to incorporate multiple implementation. (discussed on this thread: Alon's suggestion, Mark's patch for adding support for netcf etc.) - We would like to maintain at least one implementation as the working/up-to-date implementation for our users, this implementation should be distribution agnostic (as we all acknowledge this is an important goal for VDSM). I also think that with the agreement of this community we can choose to change our focus, from time to time, from one implementation to another as we see fit (today it can be OVS+netcf and in a few months we'll use the quantum based implementation if we agree it is better) 2. The second discussion is about persisting the network configuration on the host vs. dynamically retrieving it from a centralized location like the engine. Danken raised a concern that even if going with the dynamic approach the host should persist the management network configuration. About dynamical retrieving from a centralized location, when will the retrieving start? Just in the very early stage of host booting before network functions? Or after the host startup and in the normal running state of the host? Before retrieving the configuration, how does the host network connecting to the engine? I think we need a basic well known network between hosts and the engine first. Then after the retrieving, hosts should reconfigure the network for later management. However, the timing to retrieve and reconfigure are challenging. Obviously the second discussion influences the API modeling. Since I think it would be challenging to add support for generic API and change the current implementation to match the dynamic configuration approach simultaneously I suggest we'll focus our efforts on one change at a time. I suggest to have a discussion on the pro's and con's of dynamic configuration and after we get to a consensus around that we can start modeling the generic API. thoughts? comments? Livnat ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Proposal to add Adam Litke as maintainer to vdsm
+1 Itamar Heim: Adam has been submitting numerous patches for VDSM for more than a year, most noticeably on improving VDSM API into, well, an API... I'd like to propose Adam as a maintainer for vdsm. Thanks, Itamar ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Adding Local Storage Domain failed
{} resources {} Thread-53::DEBUG::2012-11-14 02:16:19,544::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-53::ERROR::2012-11-14 02:16:19,544::dispatcher::69::Storage.Dispatcher.Protect::(run) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 61, in run result = ctask.prepare(self.func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 1142, in prepare raise self.error Timeout Thread-59::DEBUG::2012-11-14 02:16:23,767::task::568::TaskManager.Task::(_updateState) Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::moving from state init -> state preparing Thread-59::INFO::2012-11-14 02:16:23,768::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-59::INFO::2012-11-14 02:16:23,768::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} Thread-59::DEBUG::2012-11-14 02:16:23,768::task::1151::TaskManager.Task::(prepare) Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::finished: {} Thread-59::DEBUG::2012-11-14 02:16:23,768::task::568::TaskManager.Task::(_updateState) Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::moving from state preparing -> state finished Thread-59::DEBUG::2012-11-14 02:16:23,768::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-59::DEBUG::2012-11-14 02:16:23,768::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-59::DEBUG::2012-11-14 02:16:23,768::task::957::TaskManager.Task::(_decref) Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::ref 0 aborting False Thread-65::DEBUG::2012-11-14 02:16:33,921::task::568::TaskManager.Task::(_updateState) Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::moving from state init -> state preparing Thread-65::INFO::2012-11-14 02:16:33,921::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-65::INFO::2012-11-14 02:16:33,921::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} Thread-65::DEBUG::2012-11-14 02:16:33,922::task::1151::TaskManager.Task::(prepare) Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::finished: {} Thread-65::DEBUG::2012-11-14 02:16:33,922::task::568::TaskManager.Task::(_updateState) Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::moving from state preparing -> state finished Thread-65::DEBUG::2012-11-14 02:16:33,922::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-65::DEBUG::2012-11-14 02:16:33,922::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-65::DEBUG::2012-11-14 02:16:33,922::task::957::TaskManager.Task::(_decref) Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::ref 0 aborting False Machine with VDSM is Running on Fedora17. Any idea? Thanks, Itzik ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] A question about the SPM operation permission in VDSM
Hi, In the VDSM code about some SPM operations like HSM.deleteImage(), It is found that VDSM doesn't check if the operation will be launched on a SPM host or not. It only checks if the storage pool is already acquired by one SPM host, but not necessary the same host as the SPM operation is delivered to. The code is like this: HSM.deleteImage() { ... HSM._spmSchedule() { self.validateSPM(spUUID) <--- Only check if the storage pool was acquired by one host, but not necessary this host } ... } So it really depends on the node management application AKA ovirt-engine to dispatch the SPM operations to the right VDSM host. And the VDSM host itself doesn't check if it is the SPM host which can execute the operation. To me, it is a bit broken. When the engine query the VDSM host who is the SPM host, it can get the right one. However, the host may be broken for some reason after the engine believes it is the SPM host and the host loses the SPM privilege, another host will take the SPM role. Then the engine continue to send the SPM operations to the broken host. As a result, the SPM operation will be launched on a non-SPM host. So I think there is a small window of racing to corrupt the VDSM hosts meta data. I think VDSM host should check if it is SPM before the SPM job is scheduled. If the host lost the SPM role already, It should fail the RPC call from the engine to let the engine to retry the operation after engine knows the failure of the former call. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Users] Is this "dd" operation harmful?
于 2012-10-14 5:15, Dan Kenigsberg: > On Thu, Oct 11, 2012 at 11:38:19PM +0800, Shu Ming wrote: >> After reading the code, every mailbox should be 4096 byte size. >> And the total mailbox size is host * 4096. Ony one host is here, so >> the total mailbox size here is 4096. why should the 'dd' operation >> read 1024000 byte which is 1000K byte much lager than 4096 here? > The controlling parameter is MAX_HOST_ID=250, not the number of current > cluster members. I am wondering if we can do some optimization here, like to read and write the block size linear to the current cluster members. > -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Users] Is this "dd" operation harmful?
After reading the code, every mailbox should be 4096 byte size. And the total mailbox size is host * 4096. Ony one host is here, so the total mailbox size here is 4096. why should the 'dd' operation read 1024000 byte which is 1000K byte much lager than 4096 here? 2012-10-11 18:54, Dan Kenigsberg: On Thu, Oct 11, 2012 at 03:44:25PM +0800, Shu Ming wrote: Hi, I found some "dd" operations were launched contiguously in my vdsm.log. Is this harmful? How was this operation caused? That's storage.storage_mailbox.SPM_MailMonitor, polling for lvextend requests. dd is used, since in the old days, vdsm did not have storage.fileUtils.DirectFile. The behavior is expected, but I cannot say that it is harmless. The mailbox should be high on http://wiki.ovirt.org/wiki/Vdsm_TODO#refactoring since forking so much is a waste, as well as using strings instead of bytearrays. Making the module as a separate, testable entity, is important, too. From vdsm.log: Dummy-51000::DEBUG::2012-10-11 15:38:57,243::__init__::1249::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/6f6d4801-7447-48ea-b516-627d83e7801e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] how engine get files from node???
于 2012-9-21 20:01, Ryan Harper: * Sheldon [2012-09-21 04:42]: Hi all, I have submitted a patch about watchdog device http://gerrit.ovirt.org/#/c/7535/ If we set 'dump' action for watchdog, qemu will generate a dump file. And I have get some feedback, how engine get these files? scp? or Will vdsm support new API to list and get these files? IMO, The new API should list and get not only dump files, but also other kinds of files. That's a good question. We know that vdsm captured core files today. I don't think there is an API mechanism to retrieve the core files remotely yet. If we did propose such a call, what sets of files would we want to include? Would this be a query for fetching debug data for a single VM, for the host (which would/could include all cores and dumps)? At least, the log files of VDSM(/var/log/vdsm/), the log files of libvirt VMs( /var/log/libvirt/qemu/), VM panic cores and Qemu crash core dumps can be included. Among them, the log files of VDSM are global ones and the others should be queried based on a specific VM. -- Sheldon Feng IBM Linux Technology Center -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] is gerrit.ovirt.org down?
It seems gerrit has downed for several times recently. Is there any special reason? 于 2012-9-12 22:45, Alon Bar-Lev: yes. - Original Message - From: "Shireesh Anjal" To: engine-de...@ovirt.org Sent: Wednesday, September 12, 2012 5:43:35 PM Subject: [Engine-devel] is gerrit.ovirt.org down? ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] GlusterFS domain specific changes
于 2012-9-7 13:21, M. Mohan Kumar 写道: On Thu, 6 Sep 2012 18:59:19 -0400 (EDT), Ayal Baron wrote: - Original Message - - Original Message - From: "M. Mohan Kumar" To: vdsm-devel@lists.fedorahosted.org Sent: Wednesday, July 25, 2012 1:26:15 PM Subject: [vdsm] [RFC] GlusterFS domain specific changes We are developing a GlusterFS server translator to export block devices as regular files to the client. Using block devices to serve VM images gives performance improvements, since it avoids some file system bottlenecks in the host kernel. Goal is to use one block device(ie file at the client side) per VM image and feed this file to QEMU to get the performance improvements. QEMU will talk to glusterfs server directly using libgfapi. Currently we support only exporting Volume groups and Logical Volumes. Logical volumes are exported as regular files to the client. Are you actually using LVM behind the scenes? If so, why bother with exposing the LVs as files and not raw block devices? Ayal, The idea is to provide a FS interface for managing block devices. One can mount the Block Device Gluster Volume and create a LV and size it just by $ touch lv1 $ truncate -s5G lv1 And other file commands can be used to clone LVs, snapshot LVs $ ln lv1 lv2 # clones $ ln -s lv1 lv1.sn # creates snapshot Do we have special reason to use "ln"? Why not use "cp" as the comannd to do the snapshot instead of "ln"? By enabling this feature GlusterFS can directly export storage in SAN. We are planning to add feature to export LUNs also as regular files in future. IMO, The major feature of GlusterFS is to export distributed local disks to the clients. If we have SAN in the backend, that means the storage block devices should be exported to clients natually. Why do we need GlusterSF to export the block devices in SAN? In GlusterFS terminology a volume capable of exporting block devices is created by specifying the 'Volume Group' (ie VG in Logical Volume management). Block Device translator(BD xlator) exports this volume group as a directory and LVs under it as regular files. In the gluster mount point creating a file results in creating a logical volume, removing a file results in removing logical volume etc. When a GlusterFS volume enabled with BD xlator is used, directory creation in that gluster mount path is not supported because directory maps to Volume groups in BD xlator. But it could be an issue in VDSM environment when a new VDSM volume is created for GlusterFS domain, VDSM mounts the storage domain and creates directories under that and create files for vm image and other uses (like meta data). Is it possible to modify this behavior in VDSM to use flat structure instead of creating directories and VM images and other files underneath it? ie for GlusterFS domain with BD xlator VDSM will not create any directory and only creates all required files under the mount point directory itself. From your description I think that the GlusterFS for block devices is actually more similar to what happens with the regular block domains. You should probably need to mount the share somewhere in the system and then use symlinks to point to the volumes. Create a regular block domain and look inside /rhev/data-center/mnt/blockSD, you'll probably get the idea of what I mean. That said we'd need to come up with a way of extending the LVs on the gluster server when required (for thin provisioning). Why? if it's exposed as a file that probably means it supports sparseness. i.e. if this becomes a new type of block domain it should only support 'preallocated' images. For start using the LVs we will always do truncate for the required size, it will resize the LV. I didn't get what you are mentioning about thin-provisioning, but I have a dumb code using dm-thin targets showing BD xlators can be extended to use dm-thin targets for thin-provisioning. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC]about the implement of text-based console
于 2012-8-31 5:26, Adam Litke 写道: On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: Hi, I submited a patch for text-based console http://gerrit.ovirt.org/#/c/7165/ the issue I want to discussing as below: 1. fix port VS dynamic port Use fix port for all VM's console. connect console with 'ssh vmUUID@ip -p port'. Distinguishing VM by vmUUID. The current implement was vdsm will allocated port for console dynamically and spawn sub-process when VM creating. In sub-process the main thread responsible for accept new connection and dispatch output of console to each connection. When new connection is coming, main processing create new thread for each new connection. Dynamic port will allocated port for each VM and use range port. It isn't good for firewall rules. so I got a suggestion that use fix port. and connect console with 'ssh vmuuid@hostip -p fixport'. this is simple for user. We need one process for accept new connection from fix port and when new connection is coming, spawn sub-process for each vm. But because the console only can open by one process, main process need responsible for dispatching console's output of all vms and all connection. So the code will be a little complex then dynamic port. So this is dynamic port VS fix port and simple code VS complex code. From a usability point of view, I think the fixed port suggestion is nicer. This means that a system administrator needs only to open one port to enable remote console access. If your initial implementation limits console access to one connection per VM would that simplify the code? Another thing we want to take care is the security. Enabling one port will make all console output accessable to the user. We should take care about this to ensure that one common user can not see the console of other vms belonging to another user. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] why password is needed in virsh after vdsm is installed?
On 2012-8-22 19:47, xiaxia347os wrote: Hello, guys after VDSM is installed, when I want to use virsh to list VMs on local host, it prompt to input password and username. What could I do to make the connection automatically succeed instead of input username and password interactively? Do you mean to just remove the password? Or make the input process automatically done by virsh? By the way, I am also wondering which authentication method is used for libvirt and vm migration in VDSM, I'd like to try these method for another usercase. SASL is used for authentication between libvirt and VDSM, also libvirt is configured by VDSM. I think what your problem here is how a host can authenticate the incoming request from the other host for VM migration to itself, not the authentication between VDSM and libvirt. I believe you can configure libvirt to use tls talking to the remote host as a certain level of authentication between hosts without inputting username and password interactively. Also, a host must have all the client certificates for every other hosts to make sure it can authenticated by the other hosts in the same cluster. Best Regards. Wenchao Xia ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Fatal error when migrate the VM with disk on iscsi data center
Hi, I am testing the VM migration in oVirt engine 3.1 bteta release and successfully did the migration test on NFS date center. However, the test of VM migration on iSCSI data center failed. I dumped the error and warning segments from the vdsm logs on both the source and destination host. Please ignore the time stamp out of sync between the two hosts. _ _*The vdsm log segment from the source host:* 66-4da5-8310-6dcc970e5367`::migration downtime thread exiting Thread-115166::ERROR::2012-07-25 23:18:55,953::vm::176::vm.Vm::(_recover) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::operation failed: Failed to connect to remote libvirt URI qemu+tcp://9.181.129.110/system Dummy-115148::DEBUG::2012-07-25 23:18:55,981::__init__::1249::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/0b9a4ea4-d487-11e1-b614-5254001498c4/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) Thread-115166::ERROR::2012-07-25 23:18:56,028::vm::240::vm.Vm::(run) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Failed to migrate Traceback (most recent call last): File "/usr/share/vdsm/vm.py", line 223, in run self._startUnderlyingMigration() File "/usr/share/vdsm/libvirtvm.py", line 451, in _startUnderlyingMigration None, maxBandwidth) File "/usr/share/vdsm/libvirtvm.py", line 491, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 82, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1034, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tcp://9.181.129.110/system *The vdsm log segment from the destination host:* Thread-1577::INFO::2012-07-25 23:19:50,407::API::601::vds::(_getNetworkIp) network None: using 0 Thread-1577::INFO::2012-07-25 23:19:50,407::API::228::vds::(create) vmContainerLock acquired by vm 11e06222-2d66-4da5-8310-6dcc970e5367 Thread-1578::DEBUG::2012-07-25 23:19:50,411::vm::564::vm.Vm::(_startUnderlyingVm) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Start Thread-1577::DEBUG::2012-07-25 23:19:50,411::API::244::vds::(create) Total desktops after creation of 11e06222-2d66-4da5-8310-6dcc970e5367 is 1 Thread-1578::DEBUG::2012-07-25 23:19:50,411::vm::568::vm.Vm::(_startUnderlyingVm) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::_ongoingCreations acquired Thread-1577::DEBUG::2012-07-25 23:19:50,412::libvirtvm::2438::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::migration destination: waiting 36s for path preparation Thread-1578::INFO::2012-07-25 23:19:50,412::libvirtvm::1285::vm.Vm::(_run) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::VM wrapper has started Thread-1578::WARNING::2012-07-25 23:19:50,413::vm::398::vm.Vm::(getConfDevices) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}' found Thread-1578::ERROR::2012-07-25 23:24:50,484::vm::604::vm.Vm::(_startUnderlyingVm) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::The vm start process failed Traceback (most recent call last): File "/usr/share/vdsm/vm.py", line 584, in _startUnderlyingVm self._waitForIncomingMigrationFinish() File "/usr/share/vdsm/libvirtvm.py", line 1572, in _waitForIncomingMigrationFinish self._connection.lookupByUUIDString(self.id), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 82, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2608, in lookupByUUIDString if ret is None:raise libvirtError('virDomainLookupByUUIDString() failed', conn=self) libvirtError: Domain not found: no domain with matching uuid '11e06222-2d66-4da5-8310-6dcc970e5367' - Timed out (did not receive success event) Thread-1578::DEBUG::2012-07-25 23:24:50,485::vm::920::vm.Vm::(setDownStatus) vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Changed state to Down: Domain not found: no domain with matching uuid '11e0622 Any idea? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Verify the storage data integrity after some storage operations with test cases
On 2012-7-17 23:21, Saggi Mizrahi wrote: Actually setting up isos and installing an OS is an overkill IMHO. Using libguestfs seems simpler as it has python bindings. What you could do is: 1. use libguest fs to format a file system on an image Do you mean the test case use libguest fs to format a file without interaction with VM? 2. Put files on said file system with libguestfs Do you mean the test case put files to the files system with libguestfs directly without interaction with VM? Or even VM is not running? 3. Snapshot If we want to test online snapshot, we should have a running VM now. 4. run fsck with libguestfs 5. rinse 6. repeast If you don't trust fsck to detect all issues you can use libguestfs to get an md5sum of the raw drive and make sure that after a snapshot it stays the same. - Original Message - From: "Shu Ming" To: "VDSM Project Development" Sent: Monday, July 16, 2012 10:28:25 PM Subject: [vdsm] Verify the storage data integrity after some storage operations with test cases Hi, To verify the storage data integrity after some storage operations like snapshot, merging by VDSM. Here are the test cases I am pondering. I would like to know your feedback about these thoughts. 1) An customized ISO image with the agent required prepared for bringing up a VM in VDSM 2) The test case will inform VDSM to create a VM from the customized ISO image 3) The test case will install an IO application to the VM 3) The test case communicate with the VDSM to inform the IO application in the VM to write some data intentionally. 4) The test case sends the commands to VDSM do some storage operation like disk snapshot, volume merging, &etc. Say snapshot operation here for an example. 5) VDSM then tell the test case the result of the operation like the name of the snapshot. 6) Test case can read the snapshot made to verify the snapshot with the data written in 3). Note: currently, there is no tool to read the snapshot image directly. We can restart the VM with the snapshot as the active disk and tell the IO application in the VM to read the data writen before for test case. And test case can compare the data read with the data it informs the application in 3). 7) If the two data matches, the storage operation succeed or it fails. In order to write such a test case, these VDSM features will be required. 1) VDSM can create a VM from a specific ISO image (Almost works) 2) Test case can install an IO application to the VM by VDSM (by ovirt-agent?) 3) Test case must have some protocols with the IO application in VM for passing the command to the VM and returning the result from the VM to the test case(by ovirt-agent?). 4) The IO application can be seen as an test agent. We may extend the existing agent like ovirt-agent as the IO application. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Verify the storage data integrity after some storage operations with test cases
Hi, To verify the storage data integrity after some storage operations like snapshot, merging by VDSM. Here are the test cases I am pondering. I would like to know your feedback about these thoughts. 1) An customized ISO image with the agent required prepared for bringing up a VM in VDSM 2) The test case will inform VDSM to create a VM from the customized ISO image 3) The test case will install an IO application to the VM 3) The test case communicate with the VDSM to inform the IO application in the VM to write some data intentionally. 4) The test case sends the commands to VDSM do some storage operation like disk snapshot, volume merging, &etc. Say snapshot operation here for an example. 5) VDSM then tell the test case the result of the operation like the name of the snapshot. 6) Test case can read the snapshot made to verify the snapshot with the data written in 3). Note: currently, there is no tool to read the snapshot image directly. We can restart the VM with the snapshot as the active disk and tell the IO application in the VM to read the data writen before for test case. And test case can compare the data read with the data it informs the application in 3). 7) If the two data matches, the storage operation succeed or it fails. In order to write such a test case, these VDSM features will be required. 1) VDSM can create a VM from a specific ISO image (Almost works) 2) Test case can install an IO application to the VM by VDSM (by ovirt-agent?) 3) Test case must have some protocols with the IO application in VM for passing the command to the VM and returning the result from the VM to the test case(by ovirt-agent?). 4) The IO application can be seen as an test agent. We may extend the existing agent like ovirt-agent as the IO application. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
rt a supportable API. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would you need? I personally think that using something like AMQP for inter-node communication and engine - node would be optimal. With a rest interface that just send messages though something like AMQP. I would also not dismiss AMQP so soon we want a bug with more than a single listener at engine side (engine, history db, maybe event correlation service). collectd as a means for statistics already supports it as well. I'm for having REST as well, but not sure as main one for a consumer like ovirt engine. I agree that a message bus could be a very useful model of communication between ovirt-engine components and multiple vdsm instances. But the complexities and dependencies of AMQP do not make it suitable for use as a low-level API. AMQP will repel new adopters. Why not establish a libvdsm that is more minimalist and can be easily used by everyone? Then AMQP brokers can be built on top of the stable API with ease. All AMQP should require of the low-level API are standard function calls and an events mechanism. Thanks Robert The current XML-RPC API contains a lot of decencies and inefficiencies and we would like to retire it as soon as we possibly can. Engine would like us to move to a message based API and 3rd parties want something simple like REST so it looks like no one actually wants to use XML-RPC. Not even us. I am proposing that AMQP brokers and REST APIs could be written against the public API. In fact, they need not even live in the vdsm tree anymore if that is what we choose. Core vdsm would only be responsible for providing libvdsm and whatever language bindings we want to support. If we take the libvdsm route, the only reason to even have a REST bridge is only to support OSes other then Linux which is something I'm not sure we care about at the moment. That might be true regarding the current in-tree implementation. However, I can almost guarantee that someone wanting to write a web GUI on top of standalone vdsm would want a REST API to talk to. But libvdsm makes this use case of no concern to the core vdsm developers. I do think that having C supportability in our API is a good idea, but the current API should not be used as the base. Let's _start_ with a schema document that describes today's API and then clean it up. I think that will work better than starting from scratch. Once my schema is written I will post it and we can 'patch' it as a community until we arrive at a 1.0 version we are all happy with. +1 Ok. Redoubling my efforts to get this done. Describing the output of list(True) takes awhile :) ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Adam Litke IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
On 2012-7-12 20:41, Adam Litke wrote: On Thu, Jul 12, 2012 at 08:11:17AM +0800, Shu Ming wrote: Basically, my understanding is that we can generate two versions of libvdsm from the schema file for both the node and the management application. First, the transportation protocols(XMLRPC, REST-API) will depend on libvdsm(node version) to export the APIs to remote management application. Secondly, the management application can use libvdsm(application version ) to emit the remote call to the node. Also, transportation protocols like REST API and XML RPC API can also be generated automatically by the schema file with C, Java, Python bindings. I think this might be a bit too complex of a model. Here's how I see it... The schema generates C/gObject code which can be compiled into libvdsm. We can use the gObject introspection library to automatically generate language bindings for Java, Python, Perl, etc. Do you mean Java, Python, Perl, etc bindings of libvdsm? The libvdsm library talks to vdsmd using a wire protocol that works locally and remotely. This wire protocol is completely hidden from library users. It's an implementation detail that can be changed later if necessary. Today I would recommend that we use xmlrpc. This means that ovirt-engine or another remote program could use libvdsm in the exact same manner as a local program. The library user just needs to call libvdsm.connect(uri). Except libvdsm.connet() or libvdsm.disconnet(), do you expect the engine will not change any of its xml-rpc interfaces now after libvdsm is introduced? I mean engine may not want to change much of its current code, so it expect the least change. Finally, REST and AMQP bridges would be written solely against libvdsm. These So we should write REST, AMQP bridges manually with Java, Python, Perl language on various bindings of libvdsm? bridges are probably not suitable for code generation (but we can revisit that as a separate issue because it's up to the bridge writer to determine the best approach). On 2012-7-12 2:29, Saggi Mizrahi wrote: I'm sorry, but I don't really understand the drawing - Original Message - From: "Shu Ming" To: "Adam Litke" Cc: vdsm-devel@lists.fedorahosted.org Sent: Wednesday, July 11, 2012 10:24:49 AM Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm Adam, Maybe, I don't fully understand your proposal. Here is my understanding of libvdsm in the picture. Please check the following link for the picture. http://www.ovirt.org/wiki/File:Libvdsm.JPG http://www.ovirt.org/wiki/File:Libvdsm.JPG On 2012-7-9 21:56, Adam Litke wrote: On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote: On 07/06/2012 01:15 AM, Robert Middleswarth wrote: On 07/05/2012 04:45 PM, Adam Litke wrote: On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote: - Original Message - From: "Adam Litke" To: "Saggi Mizrahi" Cc: "Anthony Liguori" , "VDSM Project Development" Sent: Thursday, July 5, 2012 2:34:50 PM Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote: The idea of having a supported C API was something I was thinking about doing (But I'd rather use gobject introspection and not schema generation) But the problem is not having a C API is using the current XML RPC API as it's base I want to disect this a bit to find out exactly where there might be agreement and disagreement. C API is a good thing to implement - Agreed. I also want to use gobject introspection but I don't agree that using glib precludes the use of a formalized schema. My proposal is that we write a schema definition and generate the glib C code from that schema. I agree that the _current_ xmlrpc API makes a pretty bad base from which to start a supportable API. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would you ne
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
Basically, my understanding is that we can generate two versions of libvdsm from the schema file for both the node and the management application. First, the transportation protocols(XMLRPC, REST-API) will depend on libvdsm(node version) to export the APIs to remote management application. Secondly, the management application can use libvdsm(application version ) to emit the remote call to the node. Also, transportation protocols like REST API and XML RPC API can also be generated automatically by the schema file with C, Java, Python bindings. On 2012-7-12 2:29, Saggi Mizrahi wrote: I'm sorry, but I don't really understand the drawing - Original Message - From: "Shu Ming" To: "Adam Litke" Cc: vdsm-devel@lists.fedorahosted.org Sent: Wednesday, July 11, 2012 10:24:49 AM Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm Adam, Maybe, I don't fully understand your proposal. Here is my understanding of libvdsm in the picture. Please check the following link for the picture. http://www.ovirt.org/wiki/File:Libvdsm.JPG http://www.ovirt.org/wiki/File:Libvdsm.JPG On 2012-7-9 21:56, Adam Litke wrote: On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote: On 07/06/2012 01:15 AM, Robert Middleswarth wrote: On 07/05/2012 04:45 PM, Adam Litke wrote: On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote: - Original Message - From: "Adam Litke" To: "Saggi Mizrahi" Cc: "Anthony Liguori" , "VDSM Project Development" Sent: Thursday, July 5, 2012 2:34:50 PM Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote: The idea of having a supported C API was something I was thinking about doing (But I'd rather use gobject introspection and not schema generation) But the problem is not having a C API is using the current XML RPC API as it's base I want to disect this a bit to find out exactly where there might be agreement and disagreement. C API is a good thing to implement - Agreed. I also want to use gobject introspection but I don't agree that using glib precludes the use of a formalized schema. My proposal is that we write a schema definition and generate the glib C code from that schema. I agree that the _current_ xmlrpc API makes a pretty bad base from which to start a supportable API. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would you need? I personally think that using something like AMQP for inter-node communication and engine - node would be optimal. With a rest interface that just send messages though something like AMQP. I would also not dismiss AMQP so soon we want a bug with more than a single listener at engine side (engine, history db, maybe event correlation service). collectd as a means for statistics already supports it as well. I'm for having REST as well, but not sure as main one for a consumer like ovirt engine. I agree that a message bus could be a very useful model of communication between ovirt-engine components and multiple vdsm instances. But the complexities and dependencies of AMQP do not make it suitable for use as a low-level API. AMQP will repel new adopters. Why not establish a libvdsm that is more minimalist and can be easily used by everyone? Then AMQP brokers can be built on top of the stable API with ease. All AMQP should require of the low-level API are standard function calls and an events mechanism. Thanks Robert The current XML-RPC API contains a lot of decencies and inefficiencies and we would like to retire it as soon as we possibly can. Engine would like us to move to a message based API and 3rd parties want something simple like REST so it looks like no one actually wants to use XML-RPC. Not even us. I am proposing that AMQP brokers and REST APIs could be written against the public API. In fact, they need not even live in the vdsm tree anymore if that
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
7;patch' it as a community until we arrive at a 1.0 version we are all happy with. +1 Ok. Redoubling my efforts to get this done. Describing the output of list(True) takes awhile :) ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Creating a new VM with a template
Hi, I created a VM with from existing template and found that the image of the new VM actually copied the image from the template instead of sharing a base from the template. I am wondering why the new VM doesn't use a new cow image as its image pointing to the base image in template. By that way, all the new VMs will share a common parent image in template and save the space in the storage domain. Is there any other concern not to do this? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Readonly leases error in vdsm log
On 2012-7-5 20:12, Deepak C Shetty wrote: It looks like its trying to use the cdrom disk device as a disk lea It seems this bug was hit: *Bug 828633* <https://bugzilla.redhat.com/show_bug.cgi?id=828633> -Sanlock locking failed for readonly devices https://bugzilla.redhat.com/show_bug.cgi?id=828633 -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 2012-6-27 0:18, Adam Litke wrote: On Tue, Jun 26, 2012 at 04:47:35PM +0100, Daniel P. Berrange wrote: On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote: On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: 1) Completely define the current XMLRPC API including all functions, parameters, and return values. Complex data structures can be broken down into their basic types. These are: int, str, bool, list, dict, typed-dict, enum. I have already started this process and am using Qemu's QAPI schema language. You can see that here [1]. For an example of what that looks like describing the vdsm API see this snippet [2]. 2) Import the parser/generator code from qemu for the above schema. Vdsm will require a few extensions such as typed-dictionaries, tuples, and type aliases. Adapt the generator so that it can produce a libvdsm which provides API language bindings for python, c, and java. 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely auto-generated from the schema and accompanying documentation. This can serve to model how new transports can be written. For example, an AMQP implementation can be implemented entirely outside of the vdsm project if we wished. It only needs to talk to vdsm via libvdsm. Easy as 1,2,3 :) [1] http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD [2] http://fpaste.org/rt96/ Probably more than you bargained for when asking for more info... :) Indeed! I am still at a loss why the languages take such a prominent place in your choice for an API. A good API is easily consumable by any language. I think you are both right here. A good API is easily consumed from any language, but this doesn't mean there is zero cost to starting to consume it from a client. You either way to be able to auto-generate code for the client side APIs in all your languages of choice, or even better, you want the client side APIs to be just do runtime dynamic dispatch based on published schema. Thanks for commenting! On one hand, dynamic dispatch seems attractive but I think dramatically increases complexity on both the client and server sides. Does anyone know of a prominent open source project that has been successful with dynamic dispatch? I am inclined to go with the C library approach because it is tried and tested and it fits the model of other virtualization libraries that I am familiar with. If you go down the route of writing a C based libvdsm for VDSM, then my recommendation would be to use the GObject APIs. You can then take full advantage of the GObject Introspection capabilities to have full dynamic dispatch in languages like python, perl, javascript, or full auto-generation of code in Vala, C#, etc Ahh, thanks for reminding me of this. GObject definitely seems like the way to go. I assume there are no real implications for the schema definition and that the heavy-lifting for GObject support would be limited to the C code generator. Time to take a closer look at the GObject stuff. Even if we can generate the various API bindings from the schema, we still need to change the transportation layers to connect the new APIs from the client to the VDSM services, whether or not the layer is XMLRPC, REST or any other way. I really hate to change the transportation layer from time to time to know new APIs. It is better to let the transportation layer to figure out the new API automatically and keep zero change with new APIs. I certainly wouldn't waste time writing your own code-generator for all the various languages, since that's just reinventing the wheel that GObject Introspection already provides for the most part. Agreed. I would love to avoid this! -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
heir basic types. These are: int, str, bool, list, dict, typed-dict, enum. I have already started this process and am using Qemu's QAPI schema language. You can see that here [1]. For an example of what that looks like describing the vdsm API see this snippet [2]. 2) Import the parser/generator code from qemu for the above schema. Vdsm will require a few extensions such as typed-dictionaries, tuples, and type aliases. Adapt the generator so that it can produce a libvdsm which provides API language bindings for python, c, and java. 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely auto-generated from the schema and accompanying documentation. This can serve to model how new transports can be written. For example, an AMQP implementation can be implemented entirely outside of the vdsm project if we wished. It only needs to talk to vdsm via libvdsm. Maybe API layer can load API interface definition from this schema. Then we only define the api interface at one place. :) I am not sure now. still thinking. I think the code for vdsm-shell can be auto-generated from the schema at build time. I prefer this over dynamic discovery because it is much simpler.` Easy as 1,2,3 :) [1] http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD [2] http://fpaste.org/rt96/ Probably more than you bargained for when asking for more info... :) -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmt integration
On 2012-6-25 22:14, Deepak C Shetty wrote: On 06/25/2012 07:47 AM, Shu Ming wrote: On 2012-6-25 10:10, Andrew Cathrow wrote: - Original Message - From: "Andy Grover" To: "Shu Ming" Cc: libstoragemgmt-de...@lists.sourceforge.net, engine-de...@ovirt.org, "VDSM Project Development" Sent: Sunday, June 24, 2012 10:05:45 PM Subject: Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmtintegration On 06/24/2012 07:28 AM, Shu Ming wrote: On 2012-6-23 20:40, Itamar Heim wrote: On 06/23/2012 03:09 AM, Andy Grover wrote: On 06/22/2012 04:46 PM, Itamar Heim wrote: On 06/23/2012 02:31 AM, Andy Grover wrote: On 06/18/2012 01:15 PM, Saggi Mizrahi wrote: Also, there is no mention on credentials in any part of the process. How does VDSM or the host get access to actually modify the storage array? Who holds the creds for that and how? How does the user set this up? It seems to me more natural to have the oVirt-engine use libstoragemgmt directly to allocate and export a volume on the storage array, and then pass this info to the vdsm on the node creating the vm. This answers Saggi's question about creds -- vdsm never needs array modification creds, it only gets handed the params needed to connect and use the new block device (ip, iqn, chap, lun). Is this usage model made difficult or impossible by the current software architecture? what about live snapshots? I'm not a virt guy, so extreme handwaving: vm X uses luns 1& 2 engine -> vdsm "pause vm X" that's pausing the VM. live snapshot isn't supposed to do so. Tough we don't expect to do a pausing operation to the VM when live snaphot is undergoing, the VM should be blocked on the access to specific luns for a while. The blocking time should be very short to avoid the storage IO time out in the VM. OK my mistake, we don't pause the VM during live snapshot, we block on access to the luns while snapshotting. Does this keep live snapshots working and mean ovirt-engine can use libsm to config the storage array instead of vdsm? Because that was really my main question, should we be talking about engine-libstoragemgmt integration rather than vdsm-libstoragemgmt integration. for snapshotting wouldn't we want VDSM to handle the coordination of the various atomic functions? I think VDSM-libstoragemgmt will let the storage array itself to make the snapshot and handle the coordination of the various atomic functions. VDSM should be blocked on the following access to the specific luns which are under snapshotting. I kind of agree. If snapshot is being done at the array level, then the array takes care of quiesing the I/O, taking the snapshot and allowing the I/O, why does VDSM have to worry about anything here, it should all happen transparently for VDSM, isnt it ? The only issue is the quiesing may time out the VDSM io functions if it takes a non-trivial time. Not sure if VDSM can handle all the time out gracefully. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Can somebody look why I don't have privilege to access bug #833425
On 2012-6-20 18:37, Itamar Heim wrote: On 06/20/2012 06:44 AM, Shu Ming wrote: Hi, My account was registered by IBM email account . This bug should be VDSM specific and I think we should have enough privilege to access VDSM bugs. fixed now. Thanks. Itamar. Also bug #784931 can not be accessed. I was wondering if we can have a general rule for the bug category triage? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About code segment in vdsm/storage/image.py
On 2012-6-25 15:21, Shu Ming wrote: Hi, I am reading the code of merge() function in image.py and get one question about the code. I wonder why we should get the volume parameters from the parent of the destination volume if the parent is existing instead from the destination volume itself? 1101if dstParentUUID != sd.BLANK_UUID: volParams = vols[dstParentUUID].getVolumeParams() else: 1105 volParams = dstVol.getVolumeParams() In my understanding, the volParams is different in internal volume merge and base volume merge(merging to a top most base without parent). For internal volume merge, the volParams comes from the parent of the destination volume. While for base volume merge, the volParams comes from the destination volume. Is it correct? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About code segment in vdsm/storage/image.py
Hi, I am reading the code of merge() function in image.py and get one question about the code. I wonder why we should get the volume parameters from the parent of the destination volume if the parent is existing instead from the destination volume itself? 1101if dstParentUUID != sd.BLANK_UUID: volParams = vols[dstParentUUID].getVolumeParams() else: 1105 volParams = dstVol.getVolumeParams() -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmt integration
On 2012-6-25 10:10, Andrew Cathrow wrote: - Original Message - From: "Andy Grover" To: "Shu Ming" Cc: libstoragemgmt-de...@lists.sourceforge.net, engine-de...@ovirt.org, "VDSM Project Development" Sent: Sunday, June 24, 2012 10:05:45 PM Subject: Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmt integration On 06/24/2012 07:28 AM, Shu Ming wrote: On 2012-6-23 20:40, Itamar Heim wrote: On 06/23/2012 03:09 AM, Andy Grover wrote: On 06/22/2012 04:46 PM, Itamar Heim wrote: On 06/23/2012 02:31 AM, Andy Grover wrote: On 06/18/2012 01:15 PM, Saggi Mizrahi wrote: Also, there is no mention on credentials in any part of the process. How does VDSM or the host get access to actually modify the storage array? Who holds the creds for that and how? How does the user set this up? It seems to me more natural to have the oVirt-engine use libstoragemgmt directly to allocate and export a volume on the storage array, and then pass this info to the vdsm on the node creating the vm. This answers Saggi's question about creds -- vdsm never needs array modification creds, it only gets handed the params needed to connect and use the new block device (ip, iqn, chap, lun). Is this usage model made difficult or impossible by the current software architecture? what about live snapshots? I'm not a virt guy, so extreme handwaving: vm X uses luns 1& 2 engine -> vdsm "pause vm X" that's pausing the VM. live snapshot isn't supposed to do so. Tough we don't expect to do a pausing operation to the VM when live snaphot is undergoing, the VM should be blocked on the access to specific luns for a while. The blocking time should be very short to avoid the storage IO time out in the VM. OK my mistake, we don't pause the VM during live snapshot, we block on access to the luns while snapshotting. Does this keep live snapshots working and mean ovirt-engine can use libsm to config the storage array instead of vdsm? Because that was really my main question, should we be talking about engine-libstoragemgmt integration rather than vdsm-libstoragemgmt integration. for snapshotting wouldn't we want VDSM to handle the coordination of the various atomic functions? I think VDSM-libstoragemgmt will let the storage array itself to make the snapshot and handle the coordination of the various atomic functions. VDSM should be blocked on the following access to the specific luns which are under snapshotting. Thanks -- Regards -- Andy ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmt integration
On 2012-6-23 20:40, Itamar Heim wrote: On 06/23/2012 03:09 AM, Andy Grover wrote: On 06/22/2012 04:46 PM, Itamar Heim wrote: On 06/23/2012 02:31 AM, Andy Grover wrote: On 06/18/2012 01:15 PM, Saggi Mizrahi wrote: Also, there is no mention on credentials in any part of the process. How does VDSM or the host get access to actually modify the storage array? Who holds the creds for that and how? How does the user set this up? It seems to me more natural to have the oVirt-engine use libstoragemgmt directly to allocate and export a volume on the storage array, and then pass this info to the vdsm on the node creating the vm. This answers Saggi's question about creds -- vdsm never needs array modification creds, it only gets handed the params needed to connect and use the new block device (ip, iqn, chap, lun). Is this usage model made difficult or impossible by the current software architecture? what about live snapshots? I'm not a virt guy, so extreme handwaving: vm X uses luns 1& 2 engine -> vdsm "pause vm X" that's pausing the VM. live snapshot isn't supposed to do so. Tough we don't expect to do a pausing operation to the VM when live snaphot is undergoing, the VM should be blocked on the access to specific luns for a while. The blocking time should be very short to avoid the storage IO time out in the VM. engine -> libstoragemgmt "snapshot luns 1, 2 to luns 3, 4" engine -> vdsm "snapshot running state of X to Y" engine -> vdsm "unpause vm X" if engine had any failure before this step, the VM will remain paused. i.e., we compromised the VM to take a live snapshot. engine -> vdsm "change Y to use luns 3, 4" ? -- Andy ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] refactor clientif move the api implement to sub-module
On 2012-6-21 10:35, Xu He Jie wrote: 于 2012年06月21日 03:10, Adam Litke 写道: On Wed, Jun 20, 2012 at 06:08:57PM +0800, Xu He Jie wrote: Hi, folks I am trying move api implement to sub-module, then we don't need singleton and passing clientif to anywhere. So I sent this mail to descript the idea. I think it's good for review. So I add api registeration mechanism. Other modules can register it's api implement to api layer. I try to move VM api and vm stuff(like clientif.vmContainer, etc) to a new module called vmm. the vmm.VMM is similar with hsm. It's responsiable for managing vm and register VM implement to api layer. Same with hsm, I move all storage related api to hsm, and hsm will register them to api layer. After all api move to submodule, we can rename clientif and needn't passing client to any where. :) I have already submit a rough version to gerrit: http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:wip_refactor_clientif,n,z I took a cursory look at the code you have submitted. I think you need to more thoroughly describe your design. I was particularly confused by your use of Abstract Base Classes for this scenario. Can you explain in more depth why you have done this? Is there a simpler way to accomplish what you need to do? I looked at your VMM patch and I am unsure why you need to define VMBase and VMImpl separately. It means declaring the set of functions in two separate files. Thanks for shining some more light on your methodology. Adam, Thanks for reply! I want to force people declare their API interface before they implement it. the vdsm API should be standard, stable, it won't be modified frequently. And anyone should implement the API as the definition. So I added Abstract Base Classes. The API interface must be declared at API.py, then people must implement as the API.py defined. I think it more clear for people want to implement the vdsm API. People can find the API definition from API.py and implement it as the interface definition. As example, VMBase is the vdsm VM API interface definition in API.py. VMBase is an Abstract Base Class. so VMImpl is an implement of VM API. When I register VMImpl to API layer by the function 'API.registAPI()'. 'API.registAPI()' will check VMImpl to ensure VMImpl implement VM API as definition of VMBase in API.py. If we didn't define API interface, so people will confuse what is vdsm API looks like and which implement of API is standard. Who will give the definition of the API interfaces? In my mind, the one giving the definitions may be different form the implementors. What is the format of the definitions will be like? Thanks! ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguestfs, virt-install, p2v/v2v tooling. All of these are available, but there isn't an easy way to use this tools through an API. - host management operations - vdsm already does some host level configuration (see networking e.g.) it would be good to think about extending the API to cover other areas of configuration and updates - hardware enumeration - driver level information - storage configuration (we've got a bit of a discussion going around libstoragemgmt here) - performance monitoring/debugging - is the host collecting enough information to do debug/perf analysis - can we support specific configurations of a host that optimize for specific workloads - and can we do this in the API such that third-parties can supply and maintain specific workload configurations All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Can somebody look why I don't have privilege to access bug #833425
Thanks. It worked. On 2012-6-20 18:37, Itamar Heim wrote: On 06/20/2012 06:44 AM, Shu Ming wrote: Hi, My account was registered by IBM email account . This bug should be VDSM specific and I think we should have enough privilege to access VDSM bugs. fixed now. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Can somebody look why I don't have privilege to access bug #833425
Hi, My account was registered by IBM email account . This bug should be VDSM specific and I think we should have enough privilege to access VDSM bugs. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
fs, virt-install, p2v/v2v tooling. All of these are available, but there isn't an easy way to use this tools through an API. - host management operations - vdsm already does some host level configuration (see networking e.g.) it would be good to think about extending the API to cover other areas of configuration and updates - hardware enumeration - driver level information - storage configuration (we've got a bit of a discussion going around libstoragemgmt here) - performance monitoring/debugging - is the host collecting enough information to do debug/perf analysis - can we support specific configurations of a host that optimize for specific workloads - and can we do this in the API such that third-parties can supply and maintain specific workload configurations All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] RFC: Writeup on VDSM-libstoragemgmt integration
er/pool free/used space, raid type etc. Need to make sure above info is listed in a coherent way across arrays (number of LUNs, raid type used? free/total per container/pool, per LUN?. Also need I/O statistics wherever possible. _______ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Change in vdsm[master]: Make vdsm/caps.py PEP8 clean
Dan, Thanks for your work to merge these files. I notice that there are three files conflicting with the master branch not yet merged. It is because the change these files are big and not synced and get a higher possibility to conflict with other changes. In order to save both the maintainers and the submitter's time, I think we should have a protocol to merge these big files. If the maintainer think the files are ready, the maintainer should inform the submitter to rebase the patches with the latest master head in one day(considering the time zone difference, one day is a must). Also, it is the submitter's responsibility to rebase the patches time to time, but not necessary every day. Pending files: a3cda3e Make vdsm/caps.py PEP8 cleanZhengsheng b2cb196 Make vdsm/clientIF.py PEP8 cleanZhengsheng 970c465 Make vdsm/guestIF.py PEP8 cleanZhengsheng On 2012-6-14 2:35, dan...@redhat.com wrote: Dan Kenigsberg has posted comments on this change. Change subject: Make vdsm/caps.py PEP8 clean .. Patch Set 3: I would prefer that you didn't submit this currently requires non-trivial rebase. and btw, sorry of the excessive noise of my scripted push of former patches. -- To view, visit http://gerrit.ovirt.org/4801 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I000b2e63f31d1e2ca64e05d367b6b9e1b79d2cdf Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shu Ming Gerrit-Reviewer: Dan Kenigsberg Gerrit-Reviewer: Shu Ming Gerrit-Reviewer: Zhou Zheng Sheng -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] __init__ module in VDSM?
Hi, When I check my vdsm.log, I found those logs. Based on the log format, the log should be printed by a module with name "__init__". However, after checking the whole files of VDSM workspace, I couldn't find a "__init__.py" file with line 1249. My question is where is the "__init__.py" file under VDSM? Thread-807::DEBUG::2012-06-08 00:01:10,730::__init__::1249::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) Thread-807::DEBUG::2012-06-08 00:01:10,752::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: = ''; = 0 Thread-807::DEBUG::2012-06-08 00:01:10,753::__init__::1249::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) Thread-807::DEBUG::2012-06-08 00:01:10,773::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: = ''; = 0 -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About the pep8cleaning topic branch
Hi, I am working on pep8cleaning topic branch and include twelve different files in the first batch of topic branch merging into the upstream. And I found the most challenging job was to keep the topic branch synced with the latest master head, especially PEP8 clean can change a file across all the lines of the file. So I am really hoping to narrow the time window of merging the topic branch into master. So I really appreciate that more people can help to review these patches and approve the patches. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] A Tool for PEP 8 Patches to Find Code Logic Changes
On 2012-6-7 21:26, Adam Litke wrote: On Thu, Jun 07, 2012 at 12:03:30PM +0800, Zhou Zheng Sheng wrote: Hi, Since there is no coverage report on tests in vdsm, if a PEP 8 patch passes the tests, we still not sure if there is no mistake in it. Viewing the diff reports on all the changes consumes a lot of time, and some small but fatal mistakes(like misspelling variable name) can easily be ignored by human eyes. So I have a try on the compiler module of Python. I write a tool named 'pydiff'. pydiff parses two Python scripts into Abstract Syntax Trees. These data structures can reflect the logic of the code, and pydiff performs a recursive compare on the trees. Then pydiff reports differences and the corresponding line numbers. In this way, pydiff ignores code style changes, and reports only logical changes of the code. I think this tool can save us a lot of time. After a PEP 8 patch passes vdsm tests and pydiff, I will get some confidence on the patch and it probably does not break anything in vdsm. This is a very nice tool. Thanks for sharing it. I would like to see all authors of PEP8 patches use this to check their patches for semantic correctness. This should greatly improve our ability to complete the PEP8 cleanup quickly. Yes, I agree with you. Also, we should merge this tool into vdsm as a helper for PEP8 clean work. Here is a usage example: test_o.py def foo(a, b): pass if __name__ == '__main__': A = [1, 2, 3] print (4, 5, 6), \ "over" foo(1, 2) print 'Hello World' test_n.py def foo(a, b): pass if __name__ == '__main__': A = [1, 2, 3] print (4, 5, 6), "over" fooo( 1, 2) print ('Hello ' 'World') Some differences of the files are just a matter of style. The only significant difference is the function call "foo()" is misspelled in "test_n.py". Run pydiff.py, it will report: $ python pydiff.py test_*.py 1 difference(s) first file: test_n.py second file: test_o.py ((8, 'fooo'), (8, 'foo')) This report tells us that 'fooo' in line 8 of "test_n.py" is different from 'foo' in line 8 of "test_o.py". It can also find insertions or deletions. Here is another simple example: old.py print 'Hello 1' print 'Hello 2' print 'Hello 3' print 'Hello 4' print 'Hello 5' new.py print 'Hello 1' print 'Hello 3' print 'Hello 4' print 'Hello 5' print 'Hello 5' Run pydiff: $ pydiff old.py new.py 2 difference(s) first file: old.py second file: new.py ((2, Printnl([Const('Hello 2')], None)), (2, None)) ((5, None), (5, Printnl([Const('Hello 5')], None))) Here "((2, Printnl([Const('Hello 2')], None)), (2, None))" means there is a print statement in line 2 of old.py, but no corresponding statement in new.py, so we can know the statement is deleted in new.py. "((5, None), (5, Printnl([Const('Hello 5')], None)))" means there is a print statement in line 5 of new.py, but no corresponding statement in old.py, so we can know the statement is inserted in new.py. Sometimes the change in code logic is acceptable, for example, change "aDict.has_key(Key)" into "Key in aDict". pydiff can report a difference in this case, but it is up to the user to judge whether it's acceptable. pydiff is just a tool to help you finding these changes. I hope it can be helpful for PEP 8 patch reviewers. If you find any bugs, please let me know. The script is in the attachment. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About the work of "New repository system"
On 2012-5-28 19:36, Ayal Baron wrote: - Original Message - Hi, I am noticing that Saggi post a set of patches to gerrit named "[WIP] New repository system". The latest update was post in Jan 9 and the latest update of depending patch was post in Feb 19. It seems that the patches are pending on "Refactor upgrade process to facilitate repository format conversions". I am wondering if there is any Repository format conversions is now in. We're now starting to look at these again. Do you mean "Orthogonal storage repository conversion"? http://gerrit.ovirt.org/#change,3045 The latest status shows it is still under working. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About the problems to create a topic branch in gerrit
On 2012-5-24 9:56, Anthony Liguori wrote: On 05/23/2012 08:44 PM, Shu Ming wrote: No feedback or don't understand the problem I described? On 2012-5-23 23:45, Shu Ming wrote: Hi, Recently, I am working on combining smaller pep8 clean works into one big batch. Because the smaller patches come from different folks with different change logs, when I tried to push these changes into gerrit to create a pep8 topic branch, I hope the change logs can be kept in the topic branch. However, I found I can not push the patches with others' name. Errors are like the blow. So I have two proposals to fix this. Can you share the output of: git log --format="%ae, %ce, %s" origin/master..HEAD in the branch you're trying to push? Yes. That is the branch I am trying to push. shuming@kvm-rhel-01 vdsm-pep8]$ git log --format="%ae, %ec, %s" origin/master..HEAD zhshz...@linux.vnet.ibm.com, c, clean PEP 8 problems in vdsm/guestIF.py zhshz...@linux.vnet.ibm.com, c, clean PEP 8 problems in vdsm/clientIF.py zhshz...@linux.vnet.ibm.com, c, clean PEP 8 problems in vdsm/caps.py m...@linux.vnet.ibm.com, c, Fixing pep8 in vdsm/define.py shao...@linux.vnet.ibm.com, c, change the code style of before_vm_start.py for P shao...@linux.vnet.ibm.com, c, change the code style of BindingXMLRPC.py for PEP shao...@linux.vnet.ibm.com, c, change the code style of before_vm_start.py for P shao...@linux.vnet.ibm.com, c, change the code style of SecureXMLRPCServer.py fo shao...@linux.vnet.ibm.com, c, change the code style of devicemapper.py for PEP8 m...@linux.vnet.ibm.com, c, Fixing pep8 in vdsm/hooks.py shao...@linux.vnet.ibm.com, c, change the code style of volume.py for PEP8 compl xiaw...@linux.vnet.ibm.com, c, pep8 clean vdsm/storage/hba.py The second column should be "shum...@linux.vnet.ibm.com". If it's not, I assume it's because you let someone else push to your repository or pulled from there repository. Gerrit won't let you do that. I cherry-picked others' patch to my workspace and want to preserve the change log of others'. But gerrit don't let me to push those change logs back with the patches. One way to work around this is to do the following: $ mkdir patches $ git format-patch -o patches/ origin/master $ git reset --hard origin/master $ git am -s patches/ $ rm -rf patches This will effectively change committer to be you for all of the commits. The '-s' flag for git am also adds your Signed-off-by. If it's already there, you can omit that option. Regards, Anthony Liguori I) Allow the submitter to push the patches signed off by others to gerrit. II) Let the maintainer to pull the patches from my workspace to merge the patches. III) Collapse the change logs from different folks into one log signed off by me and then push that big patch into gerrit. Others's signature will go way, bad. IV) other ideas? --- remote: Resolving deltas: 66% (26/39) remote: remote: ERROR: In commit f81413cdc87208f6a72a639798816064421c8b3a remote: ERROR: committer email address zhshz...@linux.vnet.ibm.com remote: ERROR: does not match your user account. remote: ERROR: remote: ERROR: The following addresses are currently registered: remote: ERROR: shum...@linux.vnet.ibm.com remote: ERROR: smin...@gmail.com remote: ERROR: remote: ERROR: To register an email address, please visit: remote: ERROR: http://gerrit.ovirt.org/#settings,contact remote: remote: To ssh://sm...@gerrit.ovirt.org:29418/vdsm.git ! [remote rejected] HEAD -> refs/for/master/pep8cleaning (invalid committer) error: failed to push some refs to 'ssh://sm...@gerrit.ovirt.org:29418/vdsm.git' -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About the problems to create a topic branch in gerrit
No feedback or don't understand the problem I described? On 2012-5-23 23:45, Shu Ming wrote: Hi, Recently, I am working on combining smaller pep8 clean works into one big batch. Because the smaller patches come from different folks with different change logs, when I tried to push these changes into gerrit to create a pep8 topic branch, I hope the change logs can be kept in the topic branch. However, I found I can not push the patches with others' name. Errors are like the blow. So I have two proposals to fix this. I) Allow the submitter to push the patches signed off by others to gerrit. II) Let the maintainer to pull the patches from my workspace to merge the patches. III) Collapse the change logs from different folks into one log signed off by me and then push that big patch into gerrit. Others's signature will go way, bad. IV) other ideas? --- remote: Resolving deltas: 66% (26/39) remote: remote: ERROR: In commit f81413cdc87208f6a72a639798816064421c8b3a remote: ERROR: committer email address zhshz...@linux.vnet.ibm.com remote: ERROR: does not match your user account. remote: ERROR: remote: ERROR: The following addresses are currently registered: remote: ERROR:shum...@linux.vnet.ibm.com remote: ERROR:smin...@gmail.com remote: ERROR: remote: ERROR: To register an email address, please visit: remote: ERROR: http://gerrit.ovirt.org/#settings,contact remote: remote: To ssh://sm...@gerrit.ovirt.org:29418/vdsm.git ! [remote rejected] HEAD -> refs/for/master/pep8cleaning (invalid committer) error: failed to push some refs to 'ssh://sm...@gerrit.ovirt.org:29418/vdsm.git' -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About the problems to create a topic branch in gerrit
Hi, Recently, I am working on combining smaller pep8 clean works into one big batch. Because the smaller patches come from different folks with different change logs, when I tried to push these changes into gerrit to create a pep8 topic branch, I hope the change logs can be kept in the topic branch. However, I found I can not push the patches with others' name. Errors are like the blow. So I have two proposals to fix this. I) Allow the submitter to push the patches signed off by others to gerrit. II) Let the maintainer to pull the patches from my workspace to merge the patches. III) Collapse the change logs from different folks into one log signed off by me and then push that big patch into gerrit. Others's signature will go way, bad. IV) other ideas? --- remote: Resolving deltas: 66% (26/39) remote: remote: ERROR: In commit f81413cdc87208f6a72a639798816064421c8b3a remote: ERROR: committer email address zhshz...@linux.vnet.ibm.com remote: ERROR: does not match your user account. remote: ERROR: remote: ERROR: The following addresses are currently registered: remote: ERROR:shum...@linux.vnet.ibm.com remote: ERROR:smin...@gmail.com remote: ERROR: remote: ERROR: To register an email address, please visit: remote: ERROR: http://gerrit.ovirt.org/#settings,contact remote: remote: To ssh://sm...@gerrit.ovirt.org:29418/vdsm.git ! [remote rejected] HEAD -> refs/for/master/pep8cleaning (invalid committer) error: failed to push some refs to 'ssh://sm...@gerrit.ovirt.org:29418/vdsm.git' -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Agenda for tomorrow's call
How about the new repository system latest status? http://gerrit.ovirt.org/#change,192 On 2012-5-21 3:55, Ayal Baron wrote: Hi all, I would like to discuss the following on our upcoming call: - reviewers are missing! - reviewers/verifiers are missing for pep8 patches. I would like to ask a volunteer to aggregate them all in one branch, and get some folks from Red Hat QE to run some sanity test on them. - functional tests: Wenchao Xia's http://gerrit.ovirt.org/#change,4454 and Adam Litke's http://gerrit.ovirt.org/#change,4451 - Saggi's unicode fixes to betterPopen - Stories about negative flows hurt by commit 1676396f18cf5c300d87e181 "Change safelease APIs to match SANLock flow" - Upcoming oVirt-3.1 release: when to break from master branch? Anyone else has more interesting stuff? We can skip my bullets for a few of yours if we do not have time. Regards, Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [libvirt] F17's libvirt takes comments into LIBVIRTD_ARGS
On 2012-5-16 23:35, Daniel P. Berrange wrote: On Wed, May 16, 2012 at 06:08:51PM +0300, Dan Kenigsberg wrote: On Wed, May 16, 2012 at 11:05:16PM +0800, Shu Ming wrote: On 2012-5-16 18:46, Dan Kenigsberg wrote: On Tue, May 15, 2012 at 04:16:11PM +0800, Shu Ming wrote: On 2012-5-14 7:30, Dan Kenigsberg wrote: On Sun, May 13, 2012 at 11:51:48PM +0800, Shu Ming wrote: Hi, Recently, I found that my host in engine was always in a "unassigned state" after the host node was installed. After looking into the vdsm.log, it seemed that vdsm failed to call libvirt as an error, "libvirtError: Cannot write data: Broken pipe". When I started virsh in the host node at that time, a warning was given "WARNING: no socket to connect to" and core dumped with "virsh net-list". It looks like that no right socket was created for virsh to connect to libvirtd. Any comments about this problem? The followings are my steps in the node: [root@ovirt-node1 ~]# rpm -qa |grep vdsm vdsm-cli-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-python-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-vhostmd-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-4.9.6-0.183.git107644d.fc16.shuming1336622293.x86_64 vdsm-reg-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-debug-plugin-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-faqemu-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-bootstrap-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep libvirt libvirt-daemon-0.9.11-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11-1.fc17.x86_64 libvirt-client-0.9.11-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11-1.fc17.x86_64 libvirt-python-0.9.11-1.fc17.x86_64 [root@ovirt-node1 ~]# virsh net-list WARNING: no socket to connect to Segmentation fault I think that merits a libvirt bug. please attach strace output to bugzilla. [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep vdsm root 1299 1 0 23:10 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm The command line of libvirt process is very odd - the comment that vdsm puts into /etc/sysconfig/libvirtd is somehow taken verbatim. That's bad, and may be related to Fedora 17's systemd services. Try to remove the comment and restart libvirtd to see if this is the case. The comment come from [root@ovirt-node1 ~]# cat /etc/sysconfig/libvirtd: I know that (see my text above). However, in F16 and before, comments have been stripped before being passed to commandline. Have you tested if all is well when the commment is removed? I removed the "# by vdsm " line from the config file. And restarted the libvirtd and vdsmd service. But no luck to make "virsh net-list" successful, still got "Segmentation fault", while "virsh -c qemu:///system -r" worked. Thanks. Please strace that segfault, there's a libvirt bug lying there. And thanks again for finding the vdsm/libvirt configuration problem in F17. Specifically do the following # debuginfo-install libvirt # gdb --args virsh net-list (gdb) run ...wait for crash.. (gdb) thread apply all backtrace Regards, Daniel [root@ovirt-node1 ~]# debuginfo-install libvirt Loaded plugins: langpacks, presto, refresh-packagekit enabling fedora-debuginfo enabling updates-debuginfo No debuginfo packages available to install [root@ovirt-node1 ~]# debuginfo-install libvirt --releasever=17 Loaded plugins: langpacks, presto, refresh-packagekit enabling fedora-debuginfo enabling updates-debuginfo No debuginfo packages available to install [root@ovirt-node1 ~]# -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] F17's libvirt takes comments into LIBVIRTD_ARGS
On 2012-5-16 18:46, Dan Kenigsberg wrote: On Tue, May 15, 2012 at 04:16:11PM +0800, Shu Ming wrote: On 2012-5-14 7:30, Dan Kenigsberg wrote: On Sun, May 13, 2012 at 11:51:48PM +0800, Shu Ming wrote: Hi, Recently, I found that my host in engine was always in a "unassigned state" after the host node was installed. After looking into the vdsm.log, it seemed that vdsm failed to call libvirt as an error, "libvirtError: Cannot write data: Broken pipe". When I started virsh in the host node at that time, a warning was given "WARNING: no socket to connect to" and core dumped with "virsh net-list". It looks like that no right socket was created for virsh to connect to libvirtd. Any comments about this problem? The followings are my steps in the node: [root@ovirt-node1 ~]# rpm -qa |grep vdsm vdsm-cli-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-python-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-vhostmd-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-4.9.6-0.183.git107644d.fc16.shuming1336622293.x86_64 vdsm-reg-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-debug-plugin-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-faqemu-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-bootstrap-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep libvirt libvirt-daemon-0.9.11-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11-1.fc17.x86_64 libvirt-client-0.9.11-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11-1.fc17.x86_64 libvirt-python-0.9.11-1.fc17.x86_64 [root@ovirt-node1 ~]# virsh net-list WARNING: no socket to connect to Segmentation fault I think that merits a libvirt bug. please attach strace output to bugzilla. [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep vdsm root 1299 1 0 23:10 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm The command line of libvirt process is very odd - the comment that vdsm puts into /etc/sysconfig/libvirtd is somehow taken verbatim. That's bad, and may be related to Fedora 17's systemd services. Try to remove the comment and restart libvirtd to see if this is the case. The comment come from [root@ovirt-node1 ~]# cat /etc/sysconfig/libvirtd: I know that (see my text above). However, in F16 and before, comments have been stripped before being passed to commandline. Have you tested if all is well when the commment is removed? I removed the "# by vdsm " line from the config file. And restarted the libvirtd and vdsmd service. But no luck to make "virsh net-list" successful, still got "Segmentation fault", while "virsh -c qemu:///system -r" worked. virsh # net-list Segmentation fault [root@ovirt-node1 ~]# virsh net-list Segmentation fault [root@ovirt-node1 ~]# virsh -c qemu:///system Please enter your authentication name: ^C [root@ovirt-node1 ~]# virsh -c qemu:///system -r Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh > net-list Name State Autostart - vdsm-ovirtmgmt active yes virsh > Let's see what our friends in libvir-list think. Dan. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] libvirtError: Cannot write data: Broken pipe when vdsm try to call libvirt
On 2012-5-14 7:30, Dan Kenigsberg wrote: On Sun, May 13, 2012 at 11:51:48PM +0800, Shu Ming wrote: Hi, Recently, I found that my host in engine was always in a "unassigned state" after the host node was installed. After looking into the vdsm.log, it seemed that vdsm failed to call libvirt as an error, "libvirtError: Cannot write data: Broken pipe". When I started virsh in the host node at that time, a warning was given "WARNING: no socket to connect to" and core dumped with "virsh net-list". It looks like that no right socket was created for virsh to connect to libvirtd. Any comments about this problem? The followings are my steps in the node: [root@ovirt-node1 ~]# rpm -qa |grep vdsm vdsm-cli-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-python-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-vhostmd-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-4.9.6-0.183.git107644d.fc16.shuming1336622293.x86_64 vdsm-reg-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-debug-plugin-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-faqemu-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-bootstrap-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep libvirt libvirt-daemon-0.9.11-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11-1.fc17.x86_64 libvirt-client-0.9.11-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11-1.fc17.x86_64 libvirt-python-0.9.11-1.fc17.x86_64 [root@ovirt-node1 ~]# virsh net-list WARNING: no socket to connect to Segmentation fault I think that merits a libvirt bug. please attach strace output to bugzilla. [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep vdsm root 1299 1 0 23:10 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm The command line of libvirt process is very odd - the comment that vdsm puts into /etc/sysconfig/libvirtd is somehow taken verbatim. That's bad, and may be related to Fedora 17's systemd services. Try to remove the comment and restart libvirtd to see if this is the case. The comment come from [root@ovirt-node1 ~]# cat /etc/sysconfig/libvirtd: # Override the default config file #LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf # Listen for TCP/IP connections # NB. must setup TLS/SSL keys prior to using this #LIBVIRTD_ARGS="--listen" # Override Kerberos service keytab for SASL/GSSAPI #KRB5_KTNAME=/etc/libvirt/krb5.tab # Override the QEMU/SDL default audio driver probing when # starting virtual machines using SDL graphics # # NB these have no effect for VMs using VNC, unless vnc_allow_host_audio # is enabled in /etc/libvirt/qemu.conf #QEMU_AUDIO_DRV=sdl # #SDL_AUDIODRIVER=pulse LIBVIRTD_ARGS=--listen # by vdsm DAEMON_COREFILE_LIMIT=unlimited # by vdsm /usr/share/vdsm/respawn --minlifetime 10 --daemon --masterpid /var/run/vdsm/respawn.pid /usr/share vdsm/vdsm vdsm 1919 1917 0 23:10 ?00:00:06 /usr/bin/python /usr/share/vdsm vdsm root 1940 1919 0 23:10 ?00:00:00 /usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 709dfdde-a668-4227-a206-3d8686b4cfa1 1919 root 1941 1940 0 23:10 ?00:00:00 /usr/bin/python /usr/share/vdsm/supervdsmServer.py 709dfdde-a668-4227-a206-3d8686b4cfa1 1919 root 3711 3055 0 23:22 pts/000:00:00 vim /var/log/vdsm/vdsm.log root 4358 4103 0 23:30 pts/100:00:00 grep --color=auto vdsm [ [root@ovirt-node1 ~]# ps -ef |grep libvirtd root 4421 1 2 23:31 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm root 4750 4103 0 23:31 pts/100:00:00 grep --color=auto libvirtd Thanks for testing on F17! Dan. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] libvirtError: Cannot write data: Broken pipe when vdsm try to call libvirt
Hi, Recently, I found that my host in engine was always in a "unassigned state" after the host node was installed. After looking into the vdsm.log, it seemed that vdsm failed to call libvirt as an error, "libvirtError: Cannot write data: Broken pipe". When I started virsh in the host node at that time, a warning was given "WARNING: no socket to connect to" and core dumped with "virsh net-list". It looks like that no right socket was created for virsh to connect to libvirtd. Any comments about this problem? The followings are my steps in the node: [root@ovirt-node1 ~]# rpm -qa |grep vdsm vdsm-cli-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-python-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-vhostmd-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-4.9.6-0.183.git107644d.fc16.shuming1336622293.x86_64 vdsm-reg-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-debug-plugin-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-faqemu-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-bootstrap-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep libvirt libvirt-daemon-0.9.11-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11-1.fc17.x86_64 libvirt-client-0.9.11-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11-1.fc17.x86_64 libvirt-python-0.9.11-1.fc17.x86_64 [root@ovirt-node1 ~]# virsh net-list WARNING: no socket to connect to Segmentation fault [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep vdsm root 1299 1 0 23:10 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm vdsm 1917 1 0 23:10 ?00:00:00 /bin/bash -e /usr/share/vdsm/respawn --minlifetime 10 --daemon --masterpid /var/run/vdsm/respawn.pid /usr/share vdsm/vdsm vdsm 1919 1917 0 23:10 ?00:00:06 /usr/bin/python /usr/share/vdsm vdsm root 1940 1919 0 23:10 ?00:00:00 /usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 709dfdde-a668-4227-a206-3d8686b4cfa1 1919 root 1941 1940 0 23:10 ?00:00:00 /usr/bin/python /usr/share/vdsm/supervdsmServer.py 709dfdde-a668-4227-a206-3d8686b4cfa1 1919 root 3711 3055 0 23:22 pts/000:00:00 vim /var/log/vdsm/vdsm.log root 4358 4103 0 23:30 pts/100:00:00 grep --color=auto vdsm [ [root@ovirt-node1 ~]# ps -ef |grep libvirtd root 4421 1 2 23:31 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm root 4750 4103 0 23:31 pts/100:00:00 grep --color=auto libvirtd -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About the work of "New repository system"
Hi, I am noticing that Saggi post a set of patches to gerrit named "[WIP] New repository system". The latest update was post in Jan 9 and the latest update of depending patch was post in Feb 19. It seems that the patches are pending on "Refactor upgrade process to facilitate repository format conversions". I am wondering if there is any update about these patches and if there is any new patch based on the latest VDSM code. Recently, I am trying to test these code on the latest VDSM upstream. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] gerrit problem when reviewing VDSM code
On 2012-5-8 13:24, Itamar Heim wrote: On 05/08/2012 04:50 AM, Shu Ming wrote: Hi, In my reviewing patches in gerrit for VDSM code, I found it was a bit awkward to understand the diffs without much context above or below the diffs. I hope I can read the unchanged lines around the diffs to get more information. The unchanged lines should be in folding state when I don't care about them and can be unfolded when I need more information. What do you think about this? just changed in gerrit the context to show you 'whole file' rather than '10 lines'? Yes. 10 lines don't have much meaning to the people not familiar with the file, but still have an interesting to review the code. More context will help them to review the code without patching the diffs into the original file. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] gerrit problem when reviewing VDSM code
Hi, In my reviewing patches in gerrit for VDSM code, I found it was a bit awkward to understand the diffs without much context above or below the diffs. I hope I can read the unchanged lines around the diffs to get more information. The unchanged lines should be in folding state when I don't care about them and can be unfolded when I need more information. What do you think about this? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] vds_bootstrap.py 's residency
Hi, I am checking the VDSM and ovirt-engine workspace for "vds_bootstrap.py" file. It was found that vds_bootstrap.py file was in VDSM workspace and was packaged into vdsm-bootstrap rpm package. Also, it was found that in the host installation process, host node will try to get the "vds_bootstrap.py" from the engine server. But "vds_bootstrap.py" is not included in any engine packages. Does that mean we should install vdsm-bootstrap package into engine server? Why not package this file into engine packages instead of vdsm packages? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Introducing a validation test package to vdsm
One more comment about the test package version. Most likely, the package version will be the same version as the VDSM package version. The rule we need to consider is: Should we keep the back compatibility with the VDSM files? Say allow newer version test package running on older VDSM files? Or just allow run the same version of test package and VDSM package.One more comment about the test package version. Most likely, the package version will be the same version as the VDSM package version. The rule we need to consider is: Should we keep the back compatibility with the VDSM files? Say allow newer version test package running on older VDSM files? Or just allow run the same version of test package and VDSM package. On 2012-4-26 21:57, Adam Litke wrote: On Thu, Apr 26, 2012 at 05:24:29PM +0800, wenchao xia wrote: Hello, vdsm now have UT suits for developer, but sometimes building and installation machine is not the same one, and additional check is need which is ignored at building time, so I think some test cases should be also run on target machine to check potential errors, Then I want to introduce a sub package as VT suits. Purpose: UT: for developers, more likely a white box, running on building environment. VT: for user and deployment, more likely a black box, running on product or testing environment, all known issue should be covered. Supposed approach: 1 modify building system to generate package: vdsm-VT.rpm. I would prefer the package name 'vdsm-test.rpm' and this package should include unit tests and verification tests. 2 install as an option, after install, user type "vdsm-VT" would make the test begin. The test runner should be able to run the full suite of unit tests and verification tests (with an option to run only unit tests or only verification tests). This can be the same program that we use in the build environment except that it will set the PYTHONPATH differently to target the installed files. Planned details: 1 Going to place cases in vdsm project in ./tests/VT. 2 On installation will move some useful UT cases into VT. If you follow my approach above, you would simply package the whole tests/ directory and no copying would be necessary. 3 use same framework UT used. 4 two sub dir in test/VT: user_case_test;general_test. What is the difference between these two types of tests? It is just a scratch from my mind, so I'd like hear your opinions. Thanks for the idea! Do you have a sample test for the verification test suite? Will it be your pipe deadlock test? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Installation dependency for VDSM-4.9.6-0.109 package on FC16
Yes, I tried these. It worked well except some gpg check errors which are harmless I think. On 2012-4-23 20:04, Federico Simoncelli wrote: Hi Shu, you can get sanlock and libvirt from fedora 17: # yum --releasever=17 update libvirt # yum --releasever=17 install sanlock And lvm2 from my repository: http://fsimonce.fedorapeople.org/lvm2/fedora-16/x86_64/ -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Installation dependency for VDSM-4.9.6-0.109 package on FC16
Requires: libvirt >= 0.9.10 Installed: libvirt-0.9.6-4.fc16.x86_64 (@updates) libvirt = 0.9.6-4.fc16 Available: libvirt-0.9.6-2.fc16.x86_64 (fedora) libvirt = 0.9.6-2.fc16 Available: libvirt-0.9.6-5.fc16.x86_64 (updates) libvirt = 0.9.6-5.fc16 You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest [root@ovirt-node2 ~]# -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Installing VDSM on an empty Fedora 16 machine?
It is seems the authentication get some problems. You can use this link as a install reference to disable the authentication . http://www.ovirt.org/wiki/Installing_VDSM_from_rpm On 2012-3-8 0:33, Saša Tomić wrote: On 03/07/2012 04:24 PM, Dan Kenigsberg wrote: Time to debug further in... Clues in /var/log/libvirtd.log? Do you see anything when you run libvirtd --listen from the command line? Dan. We are getting closer now, it appears that some paths are broken... [stomic@localhost ~]$ sudo tail /var/log/libvirtd.log 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:783 : driver=0x739960 name=QEMU 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:807 : registering QEMU as driver 10 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:783 : driver=0x739f60 name=LXC 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:807 : registering LXC as driver 11 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:783 : driver=0x73aa40 name=UML 2012-03-07 15:42:58.129+: 1590: debug : virRegisterDriver:807 : registering UML as driver 12 2012-03-07 15:42:58.131+: 1590: debug : virHookCheck:115 : No hook script /etc/libvirt/hooks/daemon 2012-03-07 15:42:58.131+: 1590: debug : virHookCheck:115 : No hook script /etc/libvirt/hooks/qemu 2012-03-07 15:42:58.131+: 1590: debug : virHookCheck:115 : No hook script /etc/libvirt/hooks/lxc 2012-03-07 15:42:58.134+: 1590: error : virNetTLSContextCheckCertBasicConstraints:166 : The certificate /etc/pki/vdsm/certs/vdsmcert.pem basic constraints show a CA, but we need one for a server [stomic@localhost ~]$ libvirtd --listen 2012-03-07 16:26:00.931+: 4893: info : libvirt version: 0.9.6, package: 4.fc16 (Fedora Project, 2011-12-19-20:27:37, x86-01.phx2.fedoraproject.org) 2012-03-07 16:26:00.931+: 4893: error : virNetTLSContextCheckCertFile:92 : Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory [stomic@localhost ~]$ locate cacert.pem /etc/pki/vdsm/certs/cacert.pem [stomic@localhost pki]$ ls -l /etc/pki/vdsm/certs/cacert.pem lrwxrwxrwx. 1 root root 32 Mar 6 18:15 /etc/pki/vdsm/certs/cacert.pem -> /etc/pki/vdsm/certs/vdsmcert.pem [stomic@localhost pki]$ ls -l /etc/pki/vdsm/certs/vdsmcert.pem -rw---. 1 vdsm kvm 1188 Mar 6 18:15 /etc/pki/vdsm/certs/vdsmcert.pem -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Installing VDSM on an empty Fedora 16 machine?
On 2012-3-7 21:38, Saša Tomić wrote: Hi to all, Does anybody have step-by-step instructions how to install vdsm on an empty Fedora 16 machine? I need a git version of vdsm (to compile from source), so the binary rpms will not do the work. What I have so far is: # for resolving the sanlock bug sudo yum update --enablerepo=updates-testing # installing additional packages for devel sudo yum install @development-tools sudo yum install fedora-packager sudo yum install git autoconf automake gcc gcc-c++ pyflakes rpm-build git clone http://gerrit.ovirt.org/p/vdsm.git cd vdsm ./autogen.sh --system && make rpm sudo yum -y install /home/stomic/rpmbuild/RPMS/x86_64/vdsm-*.rpm /home/stomic/rpmbuild/RPMS/noarch/vdsm-cli*.rpm vdsClient -s 0 getVdsCaps Does "vdsClient 0 getVdsCaps" without '-s" work? "-s" means secure connection. #This should show you Vdsm's response about its capabilities, but returns: # Connection to 0:54321 refused sudo vdsClient -s 0 getVdsStats # If I try as root, nothing changes: # Connection to root@localhost.localdomain:54321 refused Thank you in advance! Sasha ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Failed dependencies when installing VDSM from git
On 2012-3-4 19:29, Itzik Brown wrote: Hi, After running the following : # git clone http://gerrit.ovirt.org/p/vdsm.git # cd vdsm # git fetch http://gerrit.ovirt.org/p/vdsm refs/changes/95/1695/10&& git checkout FETCH_HEAD # cd ~/rpmbuild/RPMS/x86_64 # rpm -Uvh vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64.rpm I get the following error: error: Failed dependencies: libvirt>= 0.9.8 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 libvirt-python>= 0.9.8 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 logrotate>= 3.8.0 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 qemu-img>= 2:0.12.1.2-2.227 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 qemu-kvm>= 2:0.12.1.2-2.227 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 sanlock>= 1.8 is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 sanlock-python is needed by vdsm-4.9.4-0.95.gita6a2e31.el6.root1330852654.x86_64 The latest libvirt version from RHEVM Beta is: libvirt-0.9.4-21.el6.x86_64 I have the following channels registered: rhel-x86_64-rhev-mgmt-agent-6-beta rhel-x86_64-server-6 rhel-x86_64-server-6-beta Should I work with Fedora when working with upstream ? FC16 is more popular in developing VDSM. Thanks, Itzik ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Error using vdsClient
On 2012-2-29 1:40, Deepak C Shetty wrote: On 02/29/2012 01:01 AM, Douglas Landgraf wrote: On 02/28/2012 12:21 PM, Deepak C Shetty wrote: Hello, Trying vdsClient for the first time. I am getting the below error for any cmd I am trying to use... [root@llm65 ~]# vdsClient llm65.in.ibm.com getVGList Traceback (most recent call last): File "/usr/share/vdsm/vdsClient.py", line 1972, in code, message = commands[command][0](commandArgs) File "/usr/share/vdsm/vdsClient.py", line 524, in getVGList vgs = self.s.getVGList() File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1575, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1294, in single_request response = h.getresponse(buffering=True) File "/usr/lib64/python2.7/httplib.py", line 1027, in getresponse response.begin() File "/usr/lib64/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) BadStatusLine: '' [root@llm65 ~]# vdsClient llm65.in.ibm.com getConnectedStoragePoolsList Traceback (most recent call last): File "/usr/share/vdsm/vdsClient.py", line 1972, in code, message = commands[command][0](commandArgs) File "/usr/share/vdsm/vdsClient.py", line 1056, in getConnectedStoragePoolsList pools = self.s.getConnectedStoragePoolsList() File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1575, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1294, in single_request response = h.getresponse(buffering=True) File "/usr/lib64/python2.7/httplib.py", line 1027, in getresponse response.begin() File "/usr/lib64/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) BadStatusLine: '' Note : I am running the above on the host (llm65) & already have configured FC, storage via OE and via OE, I am able to create and run VMs also, but vdsClient fails to give the VG and SP lists. If your vdsm is running with SSL, you must pass the argument -s (SSL), like: vdsClient -s myHost command Otherwise, -s should be ignored. 'Guess i was running with vdsm deafults so ssl was on. Its hard to relate the 'BadStatusLine" to ssl issue:) `vdsClient -s llm65.in.ibm.com getConnectedStoragePoolsList` worked for me instead of -s , -s 0 also worked, as I learnt from the vdsClient manpage. Does 0 stand for localhost here ? Yes. 0 means localhost here. If you run the vdsClient in the same host as vdsmd, you can use "vdsClient 0 getVGList". Please let me know if this is your case or not. Thanks! ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] VDSM host network configuration
On 2012-2-16 0:05, Lei Li wrote: Hi, We are working on VDSM network REST APIs, to support the functions we need to get the list of configured networks. I found that VDSM network has a function 'listNetworks' in configNetwork.py. It can get and display the current configured network like this: # python configNetwork.py list Networks: ['bridge_one', 'bridge_three', 'bridge_two'] Vlans: [] Nics: ['eth0'] Bondings: [] But there are some problems with it. It can not display the defined networks after host restart, but the created config file are still there(/etc/sysconfig/network-scripts/..). Did I miss anything? Or Is there some way to avoid this? Lei, How about the real network configurations after the host reboot? You can run "brctl show" and "ifconfig all" to get those. Also, service log can be checked to know if there is any failure in the boot process Your suggestion and thoughts would be appreciated. Lei ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About the wiki page installing_VDSM_from_rpm
Hi, I was installing the VDSM packaging following this link: http://www.ovirt.org/wiki/Installing_VDSM_from_rpm It seems that the network configuration section is not correct. In this wiki page, the static IP is assigned to the host network interface like eth0, em1 and the outing interface of the bridge is attached to the same host interface. However, it didn't work in my server. After moving the static IP into the bridge script *"ifcfg-ovirtmgmt*", the network configuration is OK now. I was wondering if it was someone's typo to write this wiki. See the change in below: Add the following content into a new file named: */etc/sysconfig/network-scripts/ifcfg-ovirtmgmt*: DEVICE=ovirtmgmt TYPE=Bridge ONBOOT=yes DELAY=0 BOOTPROTO=dhcp *---> Should be changed to:* DEVICE=ovirtmgmt TYPE=Bridge ONBOOT=yes DELAY=0 BOOTPROTO=static IPADDR=XXX.XXX.XXX.XXX NETMASK=255.255.255.0 GATEWAY=XXX.XXX.XXX.XXX Add the following line into the configuration file of your out going interface (usually em1/eth0) the file is located at: */etc/sysconfig/network-scripts/ifcfg-em1* (assuming the device is em1) BRIDGE=ovirtmgmt Full Example DEVICE=em1 BOOTPROTO=static IPADDR=192.168.1.212 NETMASK=255.255.255.0 ONBOOT=yes BRIDGE=ovirtmgmt ---> should be changed to DEVICE=em1 BOOTPROTO=static ONBOOT=yes BRIDGE=ovirtmgmt -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live Merge and Storage Live Migration
Another question about the Storage Live Migration wiki page, in the Engine-VDSM flow: createVolumeCrossSD(): domain1: [base(raw)]<-[snap1(qcow2)]<-+-(VM) domain2: +-[snap2(qcow2)] blockMigrate(): domain1: [base(raw)]<-[snap1(qcow2)]<-+ domain2: +-[snap2(qcow2)]<-(VM) I think there must have a storage quiesce time between the snap2 creation and setting the VM on top of snap2. When the snap2 is being created, VM cannot access snap1. Only after VM is set on top of snap2, the quiesce period is end. On 2012-2-7 4:34, Federico Simoncelli wrote: During my endless journey back from FOSDEM (24 hours stuck in airports, taxi and trains because of one of the largest blizzards in Italy ever), I've been briefly working on the design for both live merge and storage live migration: http://www.ovirt.org/wiki/Features/Design/LiveMerge http://www.ovirt.org/wiki/Features/Design/StorageLiveMigration They're far for being complete but I'm not able to do anything better now. Please review if you're interested, the general idea should be correct even though there are several unfinished parts. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live Merge and Storage Live Migration
On 2012-2-7 20:23, Federico Simoncelli wrote: - Original Message - From: "Shu Ming" To: "Federico Simoncelli" Cc: "VDSM Project Development" Sent: Tuesday, February 7, 2012 4:10:38 AM Subject: Re: [vdsm] Live Merge and Storage Live Migration I am just curious how HSM interact with SPM after HSM get merge request no matter if they are in the same host. At the moment the engine is responsible to poll the merge status on the HSM (and restart it if failed). When it's successfully completed the deleteVolume command must be issued to the SPM. Sorry for confusing. Please let me explain my question again. In the Engine-VDSM Flow of "LiveMerge" wiki page, the engine will call the merge interface from engine to HSM to request the merge. I was wondering what HSM will do after it get the merge request. Does it call further interfaces in SPM by whatever way to make SPM to commit the real merge operation or just complete the operation by it self? If it does, how SPM will tell HSM the status of the merge operation?synchronizedly or asnchronizedly? In the future we might consider the mailbox as mean of communication between the HSM and the SPM, even though it's now enabled only for block storage domains (and if the irs.vol_extend_policy is ON). Still the entry point would be the deleteVolume call on the SPM. Does it mean that deleteVolume call will be from HSM to SPM directly instead of from engine? -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live Merge and Storage Live Migration
I am just curious how HSM interact with SPM after HSM get merge request no matter if they are in the same host. On 2012-2-7 4:34, Federico Simoncelli wrote: During my endless journey back from FOSDEM (24 hours stuck in airports, taxi and trains because of one of the largest blizzards in Italy ever), I've been briefly working on the design for both live merge and storage live migration: http://www.ovirt.org/wiki/Features/Design/LiveMerge http://www.ovirt.org/wiki/Features/Design/StorageLiveMigration They're far for being complete but I'm not able to do anything better now. Please review if you're interested, the general idea should be correct even though there are several unfinished parts. -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] oVirt Live Snapshots
Can someone explain what is "DB" in this wiki page? See, Live snapshots operation extend regular snapshots as follow: * Create a locked snapshot in DB On 2012-1-30 19:00, Federico Simoncelli wrote: Hi, oVirt, and more specifically VDSM, is currently implementing the live snapshot feature using the API/commands provided by libvirt and qemu. It would be great if you could review the design and the current open issues at: http://ovirt.org/wiki/Live_Snapshots Thank you, -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel