[vdsm] Does we really need _addTask() in taskManager.py?
Hi, I am reading the code in taskManager.py. It seems that _addTask was defined in class Taskmanager, but no else to use them. Also, class Taskmanager get a similar function queue() which is used. Does we really need _addTask() function or just remove it? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] VDSM test unit running path
Hi, Now, The VDSM test unit is expected to run in a un-installed enviornment and use hackVDSM() to fake modules importing. hackVDSM() is pretty tricky for other modules to be added because of the dependency order. I would propose to run the VDSM unit tests in a installed VDSM path which is the same path as other VDSM runnable binaries. By this way, we can abandon hackVDSM() and run the unit test normally. On the other hand, the VDSM building script installs the VDSM files to home/builder by default, it is reasonable to start the VDSM unit test after all parts of the building script finishes. Any comments about this idea? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Meeting minutes, Jan 28
于 2013-1-29 4:00, Dan Kenigsberg 写道: Following are the notes that I've taking during the call. I'm pretty sure that I'm missing something important, such as another blocker BZ, so please reply with corrections/addtions. - Adam: going to introduce Python binding for the new Vdsm API - Sharad has joined our call for the first time. Welcome! Interested in backup/restore functionality in oVirt, to facilitate integration with TSM, presumably http://www-142.ibm.com/software/products/us/en/tivostormana/ - Federico warns that live snapshot is not enough, since we lack live merge. After a year of daily backups, each qcow chain would be 365 hops deep. That's horrible. It looks like that Qemu 1.3 and libvirt 1.0.0 already supports the live block commit which is the base of live commit. Do we have plan to support live commit in next oVirt release? ovirt-3.2: - We should probably revert http://gerrit.ovirt.org/9315 (integrate zombie reaper in supervdsmServer) as it tickles a bug in multiprocessing. See http://lists.ovirt.org/pipermail/users/2013-January/011857.html for more details in the thread starting in http://lists.ovirt.org/pipermail/users/2013-January/011747.html (thanks for raising the issue and finding the culprit, Dead Horse) - Danken believes that all important el6-specific issues reported by dreyou are now handled in the master and ovirt-3.2 branches. Please report this list if he's wrong. - Federico reports that udev is going to revert to its Fedora 18alpha behavior so that we can keep our integration with it. I'd love to see the udev BZ listed in our tracker https://bugzilla.redhat.com/showdependencytree.cgi?id=881006hide_resolved=1 as I do not recall it. Federico, could you add it? We should probably require a newer udev release. - Federico reports that we must take a short-but-important patch from Lee Yarwood: http://gerrit.ovirt.org/#/c/11281/ Cross-distro: - Adam: a guy from IBM is now listing issues that block running Vdsm on Ubuntu. Yay! - Saggi suggests that IBM contribute an Ubuntu slave for Jenkins, so that each and every patch is tested not to introduce Ubuntu regressions - just as Red Hat has recently done with EL6. - Danken: yes, we have got el6-based unit-testing running. If you are still getting an unjustified X, you may need to rebase on current master. - Federico: NetworkManager 9 has bridge support, which leads the way to NM-based implementation of Vdsm's configNetwork. - Danken: Toni is working on a suggestion for refactoring configNetwork in such a way that multiple implementations (e.g. ifcfg-based, NM-based) can coexist. Stay stuned for his Wiki page on the subject. Other refactoring going on: - Vinzenz has begun breaking the horrible libvirtvm+vm to edible-size pieces. Reviews are most welcome, particularly from Mark Wu, whose http://gerrit.ovirt.org/10054/ only awaits verification tick. Functional Testing: - Adam: a guy from IBM (Zhou, I presume) is running the functional tests via Jenkins on his laptop. Yay! Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com - Adam: plans to add a functional test for the getVdsCaps verb, as he found out that an evil Vdsm developer has added values to the caps without updating the schema. - P.S. FlowID is now in for storage verbs as well (thanks, Douglas). I hope Haim is happier now. Happy coding! ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: New Storage API
2013-1-15 5:34, Ayal Baron: image and volume are overused everywhere and it would be extremely confusing to have multiple meanings to the same terms in the same system (we have image today which means virtual disk and volume which means a part of a virtual disk). Personally I don't like the distinction between image and volume done in ec2/openstack/etc seeing as they're treated as different types of entities there while the only real difference is mutability (images are read-only, volumes are read-write). To move to the industry terminology we would need to first change all references we have today to image and volume in the system (I would say also in ovirt-engine side) to align with the new meaning. Despite my personal dislike of the terms, I definitely see the value in converging on the same terminology as the rest of the industry but to do so would be an arduous task which is out of scope of this discussion imo (patches welcome though ;) Another distinction between Openstack and oVirt is how the Nova/ovirt-engine look upon storage systems. In Openstack, a stand alone storage service(Cinder) exports the raw storage block device to Nova. On the other hand, in oVirt, storage system is highly bounded with the cluster scheduling system which integrates storage sub-system, VM dispatching sub-system, ISO image sub systems. This combination make all of the sub-system integrated in a whole which is easy to deploy, but it make the sub-system more opaque and not harder to reuse and maintain. This new storage API proposal give us an opportunity to distinct these sub-systems as new components which export better, loose-coupling APIs to VDSM. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Live storage migration status in oVirt 3.2
Hi, I am reviewing the live storage migration with my oVirt environment updated from the public nightly repository two weeks ago. I found that the move button was still gray as before when the VM was up. Only after I deactivated the disk, did the button become into non-gray state. I am wondering if the live storage migration will be supported in oVirt 3.2 release or not. BTW: These patches below should enable the live storage migration already, but I can not see it enabled in my engine. http://gerrit.ovirt.org/5252 Change I91e641cb: pool: live storage migration implementation http://gerrit.ovirt.org/8105 Change subject: core: Live Storage Migration commands http://gerrit.ovirt.org/8103 Change subject: core: VDS Commands for Live Storage Migration http://gerrit.ovirt.org/8102 Change subject: core: Adding VDSM API for Live Storage Migration http://gerrit.ovirt.org/8470 Change subject: webadmin: Adding Live Storage Migration support http://gerrit.ovirt.org/8857 core: disable LiveStorageMigration on 3.1 -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] ovirt nightly FC17 public repositories broken?
Hi, Any one was encountering this error message when using yum on ovirt public repositories? http://ovirt.org/releases/nightly/rpm/Fedora/17/repodata/primary.xml.gz http://ovirt.org/releases/nightly/rpm/Fedora/17/repodata/primary.xml.gz: [Errno -1] Metadata file does not match checksum My vdsm host and engine server was installed with FC17+virt-review, so I used FC17 nightly releases to update my VDSM and engine packages. After a bit investigation, I found these issues. [root@node1-sming ovirt-nightly]# ls -lh * -rw-r--r--. 1 root root 0 Jan 7 16:47 cachecookie -rw-r--r--. 1 root root 702K Jan 7 14:22 filelists.xml.gz -rw-r--r--. 1 root root 132K Jan 7 14:22 other.xml.gz -rw-r--r--. 1 root root 279K Jan 7 14:22 primary.xml.gz -rw-r--r--. 1 root root 1.4K Jan 5 14:35 repomd.xml *---the date was always Jan 5 different from other files even http://resources.ovirt.org/releases/nightly**/rpm/Fedora/17/repodata/ showed it was Jan 7, also I tried to use wget to download this file and got the same thing.** * [root@node1-sming ovirt-nightly]# sha1sum other.xml.gz filelists.xml.gz primary.xml.gz 453559e86950af07c876931f318d1e92ab58f289 other.xml.gz 5318e237c0f4d2ea3d1ec24e82b9f9bbe9d23a31 filelists.xml.gz e667267d45b576844a5f2cb27ce7bcfb8bb9b4f0 primary.xml.gz [root@node1-sming ovirt-nightly]# cat repomd.xml |grep open-checksum open-checksum type=sha256ce6a5243ee81c124d0fe48d103174b93fabe60573e80b5851888298373d3be9e/open-checksum open-checksum type=sha256554a301c526d335ca4a7985e7fc7b0374bca8c061ff5addbd164de81996dc5a0/open-checksum open-checksum type=sha256be3f9e6860cd9a4ac4f3f5000ad503175e7ac79a43185a778d4ab732c0d8190c/open-checksum *Note: These checksum from repmod.xml were quite different from the result of sha1sum above.* Any clues here? -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] FW: Querying for and registering unknown disk images on a storage domain
2012-12-20 23:18, Morrissey, Christopher: Hi All, I've been working on a bit of functionality for the engine that will allow a user to query a domain for new disk images (GetUnregisteredImagesQuery) for which the engine was previously unaware and a separate command to register those images (ImportImageCommand). These commands will be exposed through the REST API. This functionality is needed as we are developing an extension/plugin to oVirt that will allow a NetApp storage controller to handle cloning the actual disks outside of oVirt and need to import them once they are cloned. We'll be using other existing APIs to attach the disk to the necessary VM once the disk is cloned. On the NetApp side, we'll ensure the disk is coalesced before cloning so as to avoid the issues of registering snapshots. I am just curious about how the third party tool like NetApp to make sure the disk of a running VM coalesced before cloning? By an agent in the VM to flush file-system cache out to the disk? GetUnregisteredImagesQuery will be accessible through the disks resource collection on a storage domain. A disks resource collection does not yet exist and will need to be added. To access the unregistered images, a parameter (maybe unregistered=true) would be passed. So the path to GET the unregistered disk images on a domain would be something like /api/storagedomains/f0dbcb33-69d3-4899-9352-8e8a02f01bbd/disks?unregistered=true. This will return a list of disk images that can be each used as input to the ImportImageCommand to get them added to oVirt. ImportImageCommand will be accessible through POSTing a disk to /api/disks?import=true. The disk will be added to the oVirt DB based on the information supplied and afterward would be available to attach to a VM. When querying for unregistered disk images, the GetUnregisteredImagesQuery command will use the getImagesList() VDSM command. Currently this only reports the GUIDs of all disk images in a domain. I had been using the getVolumesList() and getVolumeInfo() VDSM commands to fill in the information so that valid disk image objects could be registered in oVirt. It seems these two functions are set to be removed since they are too invasive into the internal VDSM workings. The VDSM team will need to either return more information about each disk as part of the getImagesList() function or add a new function getImageInfo() that will give the same information for a given image GUID. Here is the project proposal for floating disk in oVirt. I think unregistered images are also floating disks. http://www.ovirt.org/Features/DetailedFloatingDisk Note that much of this work had originally been submitted under patch http://gerrit.ovirt.org/#/c/9603/. After several reviews it was found to be lacking in its design and was using deprecated APIs that did not yet have replacements. I'm reworking the code now to conform to this design and asking for further input from the VDSM, core, and restapi teams to ensure we can get this done quickly and correctly as it is needed for the 3.2 release. -Chris *Chris Morrissey* Software Engineer NetApp Inc. 919.476.4428 ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- ?? Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Host bios information
After a quick review of the wiki page, it was stated that dmidecode gave too much informations. Only five fields will be displayed in the hardware tab, Manufactory, Version, Family, UUID and serial number. For Family, it is mean the CPU core's family. And it confuses me a bit with the CPU name and CPU type fields in general tab. I think we should chose the best one to characterizethe CPU type. ybronhei: Today in the Api we display general information about the host that vdsm export by getCapabilities Api. We decided to add bios information as part of the information that is displayed in UI under host's general sub-tab. To summaries the feature - We'll modify General tab to Software Information and add another tab for Hardware Information which will include all the bios data that we'll decide to gather from the host and display. Following this feature page: http://www.ovirt.org/Features/Design/HostBiosInfo for more details. All the parameters that can be displayed are mentioned in the wiki. I would greatly appreciate your comments and questions. Thanks. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] RFC: New Storage API
2012-12-11 4:36, Saggi Mizrahi:
- Original Message -
From: Adam Litke a...@us.ibm.com
To: Saggi Mizrahi smizr...@redhat.com
Cc: Shu Ming shum...@linux.vnet.ibm.com, engine-devel engine-de...@ovirt.org,
VDSM Project Development
vdsm-devel@lists.fedorahosted.org
Sent: Monday, December 10, 2012 1:39:51 PM
Subject: Re: [vdsm] RFC: New Storage API
On Thu, Dec 06, 2012 at 11:52:01AM -0500, Saggi Mizrahi wrote:
- Original Message -
From: Shu Ming shum...@linux.vnet.ibm.com
To: Saggi Mizrahi smizr...@redhat.com
Cc: VDSM Project Development
vdsm-devel@lists.fedorahosted.org, engine-devel
engine-de...@ovirt.org
Sent: Thursday, December 6, 2012 11:02:02 AM
Subject: Re: [vdsm] RFC: New Storage API
Saggi,
Thanks for sharing your thought and I get some comments below.
Saggi Mizrahi:
I've been throwing a lot of bits out about the new storage API
and
I think it's time to talk a bit.
I will purposefully try and keep implementation details away
and
concentrate about how the API looks and how you use it.
First major change is in terminology, there is no long a
storage
domain but a storage repository.
This change is done because so many things are already called
domain in the system and this will make things less confusing
for
new-commers with a libvirt background.
One other changes is that repositories no longer have a UUID.
The UUID was only used in the pool members manifest and is no
longer needed.
connectStorageRepository(repoId, repoFormat,
connectionParameters={}):
repoId - is a transient name that will be used to refer to the
connected domain, it is not persisted and doesn't have to be
the
same across the cluster.
repoFormat - Similar to what used to be type (eg. localfs-1.0,
nfs-3.4, clvm-1.2).
connectionParameters - This is format specific and will used to
tell VDSM how to connect to the repo.
Where does repoID come from? I think repoID doesn't exist before
connectStorageRepository() return. Isn't repoID a return value
of
connectStorageRepository()?
No, repoIDs are no longer part of the domain, they are just a
transient handle.
The user can put whatever it wants there as long as it isn't
already taken by another currently connected domain.
disconnectStorageRepository(self, repoId)
In the new API there are only images, some images are mutable
and
some are not.
mutable images are also called VirtualDisks
immutable images are also called Snapshots
There are no explicit templates, you can create as many images
as
you want from any snapshot.
There are 4 major image operations:
createVirtualDisk(targetRepoId, size, baseSnapshotId=None,
userData={}, options={}):
targetRepoId - ID of a connected repo where the disk will be
created
size - The size of the image you wish to create
baseSnapshotId - the ID of the snapshot you want the base the
new
virtual disk on
userData - optional data that will be attached to the new VD,
could
be anything that the user desires.
options - options to modify VDSMs default behavior
returns the id of the new VD
I think we will also need a function to check if a a VirtualDisk
is
based on a specific snapshot.
Like: isSnapshotOf(virtualDiskId, baseSnapshotID):
No, the design is that volume dependencies are an implementation
detail.
There is no reason for you to know that an image is physically a
snapshot of another.
Logical snapshots, template information, and any other information
can be set by the user by using the userData field available for
every image.
Statements like this make me start to worry about your userData
concept. It's a
sign of a bad API if the user needs to invent a custom metadata
scheme for
itself. This reminds me of the abomination that is the 'custom'
property in the
vm definition today.
In one sentence: If VDSM doesn't care about it, VDSM doesn't manage it.
userData being a void* is quite common and I don't understand why you would
thing it's a sign of a bad API.
Further more, giving the user choice about how to represent it's own metadata
and what fields it want to keep seems reasonable to me.
Especially given the fact that VDSM never reads it.
The reason we are pulling away from the current system of VDSM understanding
the extra data is that it makes that data tied to VDSMs on disk format.
VDSM on disk format has to be very stable because of clusters with multiple
VDSM versions.
Further more, since this is actually manager data it has to be tied to the
manager backward compatibility lifetime as well.
Having it be opaque to VDSM ties it to only one, simpler, support lifetime
instead of two.
Making userData being opaque gives flexibilities to the management
applications. To me, opaque userDaa can have two types at least. The
first is the userData for runtime only. The second is the userData
expected to be persisted into the metadata disk. For the first type,
the management applications can store their own data structures like
temporary task states, VDSM query caches etc. After the VDSM
Re: [vdsm] [VDSM][RFC] hsm service standalone
2012-12-5 23:55, ybronhei: On 12/03/2012 07:22 PM, Saggi Mizrahi wrote: HSM is not a package it's an application. Currently it and the rest of VDSM share the same process but they use RPC to communicate. This is done so that one day we can actually have them run as different processes. HSM is not something you import, it's a daemon you communicate with. - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Saggi Mizrahi smizr...@redhat.com Cc: Sheldon shao...@linux.vnet.ibm.com, a...@linux.vnet.ibm.com, vdsm-devel@lists.fedorahosted.org, Zheng Sheng ZS Zhou zhshz...@cn.ibm.com Sent: Monday, December 3, 2012 12:01:28 PM Subject: Re: [vdsm] [VDSM][RFC] hsm service standalone On Mon, Dec 03, 2012 at 11:35:44AM -0500, Saggi Mizrahi wrote: There are a bunch of precondition to having HSM pulled out. On simple things is that someone needs to go through storage/misc.py and utils.py and move all the code out to logical packages. There also needs to be a bit of a rearrangement of the code files so they can import the reusable code properly. I am also still very much against putting core VDSM in to site-packages. Would you elaborate on your position? Do you mind the wrong implications this may give about API stability? Contrary, It may help to split internal api and external (means rpc between vdsm to engine and rpc between vdsm and hsm), and it might be easier to understand the process flow and control the request from engine. For example, for migration you will receive vdsm request for migration, and that will initiate all the internal processes that we do during migration by sending requests to hsm service. The engine doesn't supposed to care about the internal rpc and those abilities does not supposed to be exposed via vdsm api if we don't allow those specific hsm operation from engine, but via HSM-python service that won't be available for the engine. Might be a good division. But lots of work indeed.. Fortunately, the hsm directory in vdsm is almost self-contained. hsm files are depending on some files like config.py, constants.py etc which are already in a vdsm site packages. After HSM-python package is created, both vdsm-4.1x and HSM-python-4.1.x package will depend on vdsm-python-4.1x package. vdsm-python-4.1x package might be a confusing name that time. It might be better named vdsm-common-python-4.1x. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] moving the collection of statistics to external process
于 2012-12-6 4:51, Itamar Heim 写道: On 12/05/2012 10:33 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 10:21:39PM +0200, Itamar Heim wrote: On 12/05/2012 10:16 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 09:01:24PM +0200, Itamar Heim wrote: On 12/05/2012 08:57 PM, Adam Litke wrote: On Wed, Dec 05, 2012 at 08:30:10PM +0200, Itamar Heim wrote: On 12/05/2012 04:42 PM, Adam Litke wrote: I wanted to know what do you think about it and if you have better solution to avoid initiate so many threads? And if splitting vdsm is a good idea here? In first look, my opinion is that it can help and would be nice to have vmStatisticService that runs and writes to separate log the vms status. Vdsm recently started requiring the MOM package. MOM also performs some host and guest statistics collection as part of the policy framework. I think it would be a really good idea to consolidate all stats collection into MOM. Then, all stats become usable within the policy and by vdsm for its own internal purposes. Today, MOM has one stats collection thread per VM and one thread for the host stats. It has an API for gathering the most recently collected stats which vdsm can use. isn't this what collectd (and its libvirt plugin) or pcp are already doing? Lot's of things collect statistics, but as of right now, we're using MOM and we're not yet using collectd on the host, right? I think we should have a single stats collection service and clients for it. I think mom and vdsm should get their stats from that service, rather than have either beholden to any new stats something needs to collect. How would this work for collecting guest statistics? Would we require collectd to be installed in all guests running under oVirt? my understanding is collectd is installed on the host, and uses collects libvirt plugin to collect guests statistics? Yes, but some statistics can only be collected by making a call to the oVirt guest agent (eg. guest memory statistics). The logical next step would be to write a collectd plugin for ovirt-guest-agent, but vdsm owns the connections to the guest agents and probably does not want to multiplex those connections for many reasons (security being the main one). and some will come from qemu-ga which libvirt will support? maybe a collectd vdsm plugin for the guest agent stats? I am thinking to have the collectd as a stand alone service to collect the statics from both ovirt-guest and qemu-ga. Then collected can export the information to host proc file system in layered architecture. Then mom or other vdsm service can get the information from the proc file system like other OS statics exported in the host. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] RFC: New Storage API
optimized, degraded, or broken.
Optimized means that the image is available and you can run VMs of it.
Degraded means that the image is available and will run VMs but it might be a
better way VDSM can represent the underlying data.
What does the represent mean here?
Broken means that the image can't be used at the moment, probably because not
all the data has been set up on the volume.
Apart from that VDSM will also return the last persisted status information
which will conatin
hostID - the last host to try and optimize of fix the image
Any host can optimize the image? No need to be SDM?
stage - X/Y (eg. 1/10) the last persisted stage of the fix.
percent_complete - -1 or 0-100, the last persisted completion percentage of the
aforementioned stage. -1 means that no progress is available for that operation.
last_error - This will only be filled if the operation failed because of
something other then IO or a VDSM crash for obvious reasons.
It will usually be set if the task was manually stopped
The user can either be satisfied with that information or as the host specified
in host ID if it is still working on that image by checking it's running tasks.
So we need a function to know what tasks are running on the image
checkStorageRepository(self, repositoryId, options={}):
A method to go over a storage repository and scan for any existing problems.
This includes degraded\broken images and deleted images that have no yet been
physically deleted\merged.
It returns a list of Fix objects.
Fix objects come in 4 types:
clean - cleans data, run them to get more space.
optimize - run them to optimize a degraded image
merge - Merges two images together. Doing this sometimes
makes more images ready optimizing or cleaning.
The reason it is different from optimize is that
unmerged images are considered optimized.
mend - mends a broken image
The user can read these types and prioritize fixes. Fixes also contain opaque
FIX data and they should be sent as received to
fixStorageRepository(self, repositoryId, fix, options={}):
That will start a fix operation.
All major operations automatically start the appropriate Fix to bring the
created object to an optimize\degraded state (the one that is quicker) unless one of the
options is
AutoFix=False. This is only useful for repos that might not be able to create
volumes on all hosts (SDM) but would like to have the actual IO distributed in
the cluster.
Other common options is the strategy option:
It has currently 2 possible values
space and performance - In case VDSM has 2 ways of completing the same
operation it will tell it to value one over the other. For example, whether to
copy all the data or just create a qcow based of a snapshot.
The default is space.
You might have also noticed that it is never explicitly specified where to look
for existing images. This is done purposefully, VDSM will always look in all
connected repositories for existing objects.
For very large setups this might be problematic. To mitigate the problem you
have these options:
participatingRepositories=[repoId, ...] which tell VDSM to narrow the search to
just these repositories
and
imageHints={imgId: repoId} which will force VDSM to look for those image ID
just in those repositories and fail if it doesn't find them there.
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
--
---
舒明 Shu Ming
Open Virtualization Engineerning; CSTL, IBM Corp.
Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or
shum...@linux.vnet.ibm.com
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [VDSM][RFC] hsm service standalone
Saggi, It might be that the wiki page was a bit confusing. Let me articulate it again and help others to understand the intention of this project. 1) Now, HSM files here refers to the files under vdsm/storage which are storage service related code. And other VDSM files call into those functions in HSM files in local way(different from RPC). So in the production environment, HSM files are linked in the VDSM process and there is no additional HSM process standing in the ovirt node. 2) This project goal is to move all the HSM files into self-contained workspace and a new stand alone HSM process will be started from these HSM files. Also a new HSM package will be built from this new self-contained workspace. Also all of the HSM APIs are public to the original VDSM service by RPC calls. It is possible that the HSM standalone workspace and the original VDSM workspace may share some functions in misc.py,task.pyetc. We can move those common files to vdsm site packages as the other common files. After this goal is achieved, we have one more HSM process in production environment and one more release package for HSM files. For backward compatibility, vdsm will run in two possible modes. VDSM service will try to discover if HSM RPC APIs are available from the new HSM process. If the discovery is successful, VDSM service will make RPC calls to the HSM process. If the discovery fails, it will fall back the legacy way calling the HSM functions by local way. 3) In order to achieve this goal, we should remove the preconditions gradually and safely. The first step here is the patch http://gerrit.ovirt.org/#/c/9345/ That try to decouple the HSM configuration parameters from the vdsm.conf and store them into a new different conf file. Saggi Mizrahi: HSM is not a package it's an application. Currently it and the rest of VDSM share the same process but they use RPC to communicate. This is done so that one day we can actually have them run as different processes. HSM is not something you import, it's a daemon you communicate with. - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Saggi Mizrahi smizr...@redhat.com Cc: Sheldon shao...@linux.vnet.ibm.com, a...@linux.vnet.ibm.com, vdsm-devel@lists.fedorahosted.org, Zheng Sheng ZS Zhou zhshz...@cn.ibm.com Sent: Monday, December 3, 2012 12:01:28 PM Subject: Re: [vdsm] [VDSM][RFC] hsm service standalone On Mon, Dec 03, 2012 at 11:35:44AM -0500, Saggi Mizrahi wrote: There are a bunch of precondition to having HSM pulled out. On simple things is that someone needs to go through storage/misc.py and utils.py and move all the code out to logical packages. There also needs to be a bit of a rearrangement of the code files so they can import the reusable code properly. I am also still very much against putting core VDSM in to site-packages. Would you elaborate on your position? Do you mind the wrong implications this may give about API stability? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [VDSM][RFC] hsm service standalone
Added Saggi, Ayal, Federico into this loop. And got two comments below. Sheldon: there is already a wiki page about hsm service standalone. http://wiki.ovirt.org/HSM_service_stand_alone And now I'm trying to split hsm as a standalone rpm according to the wiki proposal. as the wiki describe: V)The configuration parameters in vdsm.conf should be pull out to another hsm.conf file. And all VDSM process should not read hsm.conf directly. If it is necessary to get the HSM configurations, an API will be added into VDSM process for hsm configuration querying I have split the the vdsm/config.py and submit a patch http://gerrit.ovirt.org/#/c/9345/ I moved 'irs'section from vdsm/config.py to a new vdsm/storage/config.py for hsm But some of vdsm python module need to get configure items form 'irs'section, such as vdsm/clientIF.py, vdsm/libvirtvm.py. I moved vdsm/storage/config.py to site-packages/hsm when install hsm. And I import all the items form 'irs'section, by from hsm import config as hsmconfig Is it a good way to split the config.py? Also I have split the the constants.py and submit a patch http://gerrit.ovirt.org/#/c/9348/ And should I create a new hsm user for the DISKIMAGE_USER instead of vdsm user? as the wiki describe: VII) All of the python modules shared by both VDSM service and HSM service should go into python site-packages directory which is like /usr/lib/python2.7/site-packages/vdsm Now, task.py doesn't sit in this directory, we should fix it in this proposal. Task model is only for storage service now. We have plan to make task models to all VDSM service. So it is reasonable to move task.py into site-packages directory. about the site-packages directory, there are three proposals. 1. VDSM service and HSM service share the same site-packages directory, still used site-packages/vdsm and that means there will be still a vdsm-python rpm, and both vdsm and hsm depends on them. 2. VDSM service and HSM service share the same site-packages directory, but it will renamed as used site-packages/hsm Now most codes of hsm and vdsm import modules from site-packages vdsm, so all of they should be fixed. 3. VDSM service depend on vdsm-python rpm, and HSM service depend on hsm-python rpm. I need to re-sort the public modules to hsm-python or vdsm-python rpm. such as fileUtils.py, misc.py, BetterPopen and qemuImg.py which is better of the above three? Packaging HSM site package files into a another new package other than vdsm-python package doesn't prevent those files contained into a same targeting package. So at least we can do, two different package say, vdsm-core-python, vdsm-storage-python and the files in them are targeting the same directory /usr/lib/python2.7/site-packages/vdsm. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Future of Vdsm network configuration - Thread mid-summary
Livnat, Thanks for your summary. I got comments below. 2012-11-25 18:53, Livnat Peer: Hi All, We have been discussing $subject for a while and I'd like to summarized what we agreed and disagreed on thus far. The way I see it there are two related discussions: 1. Getting VDSM networking stack to be distribution agnostic. - We are all in agreement that VDSM API should be generic enough to incorporate multiple implementation. (discussed on this thread: Alon's suggestion, Mark's patch for adding support for netcf etc.) - We would like to maintain at least one implementation as the working/up-to-date implementation for our users, this implementation should be distribution agnostic (as we all acknowledge this is an important goal for VDSM). I also think that with the agreement of this community we can choose to change our focus, from time to time, from one implementation to another as we see fit (today it can be OVS+netcf and in a few months we'll use the quantum based implementation if we agree it is better) 2. The second discussion is about persisting the network configuration on the host vs. dynamically retrieving it from a centralized location like the engine. Danken raised a concern that even if going with the dynamic approach the host should persist the management network configuration. About dynamical retrieving from a centralized location, when will the retrieving start? Just in the very early stage of host booting before network functions? Or after the host startup and in the normal running state of the host? Before retrieving the configuration, how does the host network connecting to the engine? I think we need a basic well known network between hosts and the engine first. Then after the retrieving, hosts should reconfigure the network for later management. However, the timing to retrieve and reconfigure are challenging. Obviously the second discussion influences the API modeling. Since I think it would be challenging to add support for generic API and change the current implementation to match the dynamic configuration approach simultaneously I suggest we'll focus our efforts on one change at a time. I suggest to have a discussion on the pro's and con's of dynamic configuration and after we get to a consensus around that we can start modeling the generic API. thoughts? comments? Livnat ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Adding Local Storage Domain failed
)
Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::finished: {}
Thread-59::DEBUG::2012-11-14
02:16:23,768::task::568::TaskManager.Task::(_updateState)
Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::moving from state preparing -
state finished
Thread-59::DEBUG::2012-11-14
02:16:23,768::resourceManager::809::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-59::DEBUG::2012-11-14
02:16:23,768::resourceManager::844::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-59::DEBUG::2012-11-14
02:16:23,768::task::957::TaskManager.Task::(_decref)
Task=`2e64438f-64c1-4f55-82bc-1f871bba1c56`::ref 0 aborting False
Thread-65::DEBUG::2012-11-14
02:16:33,921::task::568::TaskManager.Task::(_updateState)
Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::moving from state init - state
preparing
Thread-65::INFO::2012-11-14 02:16:33,921::logUtils::37::dispatcher::(wrapper)
Run and protect: repoStats(options=None)
Thread-65::INFO::2012-11-14 02:16:33,921::logUtils::39::dispatcher::(wrapper)
Run and protect: repoStats, Return response: {}
Thread-65::DEBUG::2012-11-14
02:16:33,922::task::1151::TaskManager.Task::(prepare)
Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::finished: {}
Thread-65::DEBUG::2012-11-14
02:16:33,922::task::568::TaskManager.Task::(_updateState)
Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::moving from state preparing -
state finished
Thread-65::DEBUG::2012-11-14
02:16:33,922::resourceManager::809::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-65::DEBUG::2012-11-14
02:16:33,922::resourceManager::844::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-65::DEBUG::2012-11-14
02:16:33,922::task::957::TaskManager.Task::(_decref)
Task=`2f5bd3d8-ada8-4931-a76c-fff7dad79d3d`::ref 0 aborting False
Machine with VDSM is Running on Fedora17.
Any idea?
Thanks,
Itzik
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
--
---
舒明 Shu Ming
Open Virtualization Engineerning; CSTL, IBM Corp.
Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or
shum...@linux.vnet.ibm.com
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] A question about the SPM operation permission in VDSM
Hi,
In the VDSM code about some SPM operations like HSM.deleteImage(), It is
found that VDSM doesn't check if the operation will be launched on a SPM
host or not. It only checks if the storage pool is already acquired by
one SPM host, but not necessary the same host as the SPM operation is
delivered to. The code is like this:
HSM.deleteImage()
{
...
HSM._spmSchedule()
{
self.validateSPM(spUUID) --- Only check if the storage pool was
acquired by one host, but not necessary this host
}
...
}
So it really depends on the node management application AKA ovirt-engine
to dispatch the SPM operations to the right VDSM host. And the VDSM host
itself doesn't check if it is the SPM host which can execute the
operation. To me, it is a bit broken. When the engine query the VDSM
host who is the SPM host, it can get the right one. However, the host
may be broken for some reason after the engine believes it is the SPM
host and the host loses the SPM privilege, another host will take the
SPM role. Then the engine continue to send the SPM operations to the
broken host. As a result, the SPM operation will be launched on a
non-SPM host. So I think there is a small window of racing to corrupt
the VDSM hosts meta data. I think VDSM host should check if it is SPM
before the SPM job is scheduled. If the host lost the SPM role already,
It should fail the RPC call from the engine to let the engine to retry
the operation after engine knows the failure of the former call.
--
---
舒明 Shu Ming
Open Virtualization Engineerning; CSTL, IBM Corp.
Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or
shum...@linux.vnet.ibm.com
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Users] Is this dd operation harmful?
于 2012-10-14 5:15, Dan Kenigsberg: On Thu, Oct 11, 2012 at 11:38:19PM +0800, Shu Ming wrote: After reading the code, every mailbox should be 4096 byte size. And the total mailbox size is host * 4096. Ony one host is here, so the total mailbox size here is 4096. why should the 'dd' operation read 1024000 byte which is 1000K byte much lager than 4096 here? The controlling parameter is MAX_HOST_ID=250, not the number of current cluster members. I am wondering if we can do some optimization here, like to read and write the block size linear to the current cluster members. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Users] Is this dd operation harmful?
After reading the code, every mailbox should be 4096 byte size. And the total mailbox size is host * 4096. Ony one host is here, so the total mailbox size here is 4096. why should the 'dd' operation read 1024000 byte which is 1000K byte much lager than 4096 here? 2012-10-11 18:54, Dan Kenigsberg: On Thu, Oct 11, 2012 at 03:44:25PM +0800, Shu Ming wrote: Hi, I found some dd operations were launched contiguously in my vdsm.log. Is this harmful? How was this operation caused? That's storage.storage_mailbox.SPM_MailMonitor, polling for lvextend requests. dd is used, since in the old days, vdsm did not have storage.fileUtils.DirectFile. The behavior is expected, but I cannot say that it is harmless. The mailbox should be high on http://wiki.ovirt.org/wiki/Vdsm_TODO#refactoring since forking so much is a waste, as well as using strings instead of bytearrays. Making the module as a separate, testable entity, is important, too. From vdsm.log: Dummy-51000::DEBUG::2012-10-11 15:38:57,243::__init__::1249::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/6f6d4801-7447-48ea-b516-627d83e7801e/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None) -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] is gerrit.ovirt.org down? eom
It seems gerrit has downed for several times recently. Is there any special reason? 于 2012-9-12 22:45, Alon Bar-Lev: yes. - Original Message - From: Shireesh Anjal san...@redhat.com To: engine-de...@ovirt.org Sent: Wednesday, September 12, 2012 5:43:35 PM Subject: [Engine-devel] is gerrit.ovirt.org down? eom ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC]about the implement of text-based console
于 2012-8-31 5:26, Adam Litke 写道: On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: Hi, I submited a patch for text-based console http://gerrit.ovirt.org/#/c/7165/ the issue I want to discussing as below: 1. fix port VS dynamic port Use fix port for all VM's console. connect console with 'ssh vmUUID@ip -p port'. Distinguishing VM by vmUUID. The current implement was vdsm will allocated port for console dynamically and spawn sub-process when VM creating. In sub-process the main thread responsible for accept new connection and dispatch output of console to each connection. When new connection is coming, main processing create new thread for each new connection. Dynamic port will allocated port for each VM and use range port. It isn't good for firewall rules. so I got a suggestion that use fix port. and connect console with 'ssh vmuuid@hostip -p fixport'. this is simple for user. We need one process for accept new connection from fix port and when new connection is coming, spawn sub-process for each vm. But because the console only can open by one process, main process need responsible for dispatching console's output of all vms and all connection. So the code will be a little complex then dynamic port. So this is dynamic port VS fix port and simple code VS complex code. From a usability point of view, I think the fixed port suggestion is nicer. This means that a system administrator needs only to open one port to enable remote console access. If your initial implementation limits console access to one connection per VM would that simplify the code? Another thing we want to take care is the security. Enabling one port will make all console output accessable to the user. We should take care about this to ensure that one common user can not see the console of other vms belonging to another user. -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shum...@cn.ibm.com or shum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] why password is needed in virsh after vdsm is installed?
On 2012-8-22 19:47, xiaxia347os wrote: Hello, guys after VDSM is installed, when I want to use virsh to list VMs on local host, it prompt to input password and username. What could I do to make the connection automatically succeed instead of input username and password interactively? Do you mean to just remove the password? Or make the input process automatically done by virsh? By the way, I am also wondering which authentication method is used for libvirt and vm migration in VDSM, I'd like to try these method for another usercase. SASL is used for authentication between libvirt and VDSM, also libvirt is configured by VDSM. I think what your problem here is how a host can authenticate the incoming request from the other host for VM migration to itself, not the authentication between VDSM and libvirt. I believe you can configure libvirt to use tls talking to the remote host as a certain level of authentication between hosts without inputting username and password interactively. Also, a host must have all the client certificates for every other hosts to make sure it can authenticated by the other hosts in the same cluster. Best Regards. Wenchao Xia ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail:shum...@cn.ibm.com orshum...@linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Fatal error when migrate the VM with disk on iscsi data center
Hi,
I am testing the VM migration in oVirt engine 3.1 bteta release and
successfully did the migration test on NFS date center. However, the
test of VM migration on iSCSI data center failed.
I dumped the error and warning segments from the vdsm logs on both the
source and destination host. Please ignore the time stamp out of sync
between the two hosts.
_
_*The vdsm log segment from the source host:*
66-4da5-8310-6dcc970e5367`::migration downtime thread exiting
Thread-115166::ERROR::2012-07-25
23:18:55,953::vm::176::vm.Vm::(_recover)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::operation failed: Failed to
connect to remote libvirt URI qemu+tcp://9.181.129.110/system
Dummy-115148::DEBUG::2012-07-25
23:18:55,981::__init__::1249::Storage.Misc.excCmd::(_log) 'dd
if=/rhev/data-center/0b9a4ea4-d487-11e1-b614-5254001498c4/mastersd/dom_md/inbox
iflag=direct,fullblock count=1 bs=1024000' (cwd None)
Thread-115166::ERROR::2012-07-25 23:18:56,028::vm::240::vm.Vm::(run)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Failed to migrate
Traceback (most recent call last):
File /usr/share/vdsm/vm.py, line 223, in run
self._startUnderlyingMigration()
File /usr/share/vdsm/libvirtvm.py, line 451, in
_startUnderlyingMigration
None, maxBandwidth)
File /usr/share/vdsm/libvirtvm.py, line 491, in f
ret = attr(*args, **kwargs)
File /usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py,
line 82, in wrapper
ret = f(*args, **kwargs)
File /usr/lib64/python2.7/site-packages/libvirt.py, line 1034, in
migrateToURI2
if ret == -1: raise libvirtError ('virDomainMigrateToURI2()
failed', dom=self)
libvirtError: operation failed: Failed to connect to remote libvirt URI
qemu+tcp://9.181.129.110/system
*The vdsm log segment from the destination host:*
Thread-1577::INFO::2012-07-25
23:19:50,407::API::601::vds::(_getNetworkIp) network None: using 0
Thread-1577::INFO::2012-07-25 23:19:50,407::API::228::vds::(create)
vmContainerLock acquired by vm 11e06222-2d66-4da5-8310-6dcc970e5367
Thread-1578::DEBUG::2012-07-25
23:19:50,411::vm::564::vm.Vm::(_startUnderlyingVm)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Start
Thread-1577::DEBUG::2012-07-25 23:19:50,411::API::244::vds::(create)
Total desktops after creation of 11e06222-2d66-4da5-8310-6dcc970e5367 is 1
Thread-1578::DEBUG::2012-07-25
23:19:50,411::vm::568::vm.Vm::(_startUnderlyingVm)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::_ongoingCreations acquired
Thread-1577::DEBUG::2012-07-25
23:19:50,412::libvirtvm::2438::vm.Vm::(waitForMigrationDestinationPrepare)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::migration destination:
waiting 36s for path preparation
Thread-1578::INFO::2012-07-25
23:19:50,412::libvirtvm::1285::vm.Vm::(_run)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::VM wrapper has started
Thread-1578::WARNING::2012-07-25
23:19:50,413::vm::398::vm.Vm::(getConfDevices)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Unknown type found, device:
'{'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address':
{'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}'
found
Thread-1578::ERROR::2012-07-25
23:24:50,484::vm::604::vm.Vm::(_startUnderlyingVm)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::The vm start process failed
Traceback (most recent call last):
File /usr/share/vdsm/vm.py, line 584, in _startUnderlyingVm
self._waitForIncomingMigrationFinish()
File /usr/share/vdsm/libvirtvm.py, line 1572, in
_waitForIncomingMigrationFinish
self._connection.lookupByUUIDString(self.id),
File /usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py,
line 82, in wrapper
ret = f(*args, **kwargs)
File /usr/lib64/python2.7/site-packages/libvirt.py, line 2608, in
lookupByUUIDString
if ret is None:raise libvirtError('virDomainLookupByUUIDString()
failed', conn=self)
libvirtError: Domain not found: no domain with matching uuid
'11e06222-2d66-4da5-8310-6dcc970e5367' - Timed out (did not receive
success event)
Thread-1578::DEBUG::2012-07-25
23:24:50,485::vm::920::vm.Vm::(setDownStatus)
vmId=`11e06222-2d66-4da5-8310-6dcc970e5367`::Changed state to Down:
Domain not found: no domain with matching uuid '11e0622
Any idea?
--
Shu Ming shum...@linux.vnet.ibm.com
IBM China Systems and Technology Laboratory
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Verify the storage data integrity after some storage operations with test cases
On 2012-7-17 23:21, Saggi Mizrahi wrote: Actually setting up isos and installing an OS is an overkill IMHO. Using libguestfs seems simpler as it has python bindings. What you could do is: 1. use libguest fs to format a file system on an image Do you mean the test case use libguest fs to format a file without interaction with VM? 2. Put files on said file system with libguestfs Do you mean the test case put files to the files system with libguestfs directly without interaction with VM? Or even VM is not running? 3. Snapshot If we want to test online snapshot, we should have a running VM now. 4. run fsck with libguestfs 5. rinse 6. repeast If you don't trust fsck to detect all issues you can use libguestfs to get an md5sum of the raw drive and make sure that after a snapshot it stays the same. - Original Message - From: Shu Ming shum...@linux.vnet.ibm.com To: VDSM Project Development vdsm-devel@lists.fedorahosted.org Sent: Monday, July 16, 2012 10:28:25 PM Subject: [vdsm] Verify the storage data integrity after some storage operations with test cases Hi, To verify the storage data integrity after some storage operations like snapshot, merging by VDSM. Here are the test cases I am pondering. I would like to know your feedback about these thoughts. 1) An customized ISO image with the agent required prepared for bringing up a VM in VDSM 2) The test case will inform VDSM to create a VM from the customized ISO image 3) The test case will install an IO application to the VM 3) The test case communicate with the VDSM to inform the IO application in the VM to write some data intentionally. 4) The test case sends the commands to VDSM do some storage operation like disk snapshot, volume merging, etc. Say snapshot operation here for an example. 5) VDSM then tell the test case the result of the operation like the name of the snapshot. 6) Test case can read the snapshot made to verify the snapshot with the data written in 3). Note: currently, there is no tool to read the snapshot image directly. We can restart the VM with the snapshot as the active disk and tell the IO application in the VM to read the data writen before for test case. And test case can compare the data read with the data it informs the application in 3). 7) If the two data matches, the storage operation succeed or it fails. In order to write such a test case, these VDSM features will be required. 1) VDSM can create a VM from a specific ISO image (Almost works) 2) Test case can install an IO application to the VM by VDSM (by ovirt-agent?) 3) Test case must have some protocols with the IO application in VM for passing the command to the VM and returning the result from the VM to the test case(by ovirt-agent?). 4) The IO application can be seen as an test agent. We may extend the existing agent like ovirt-agent as the IO application. -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would you need? I personally think that using something like AMQP for inter-node communication and engine - node would be optimal. With a rest interface that just send messages though something like AMQP. I would also not dismiss AMQP so soon we want a bug with more than a single listener at engine side (engine, history db, maybe event correlation service). collectd as a means for statistics already supports it as well. I'm for having REST as well, but not sure as main one for a consumer like ovirt engine. I agree that a message bus could be a very useful model of communication between ovirt-engine components and multiple vdsm instances. But the complexities and dependencies of AMQP do not make it suitable for use as a low-level API. AMQP will repel new adopters. Why not establish a libvdsm that is more minimalist and can be easily used by everyone? Then AMQP brokers can be built on top of the stable API with ease. All AMQP should require of the low-level API are standard function calls and an events mechanism. Thanks Robert The current XML-RPC API contains a lot of decencies and inefficiencies and we would like to retire it as soon as we possibly can. Engine would like us to move to a message based API and 3rd parties want something simple like REST so it looks like no one actually wants to use XML-RPC. Not even us. I am proposing that AMQP brokers and REST APIs could be written against the public API. In fact, they need not even live in the vdsm tree anymore if that is what we choose. Core vdsm would only be responsible for providing libvdsm and whatever language bindings we want to support. If we take the libvdsm route, the only reason to even have a REST bridge is only to support OSes other then Linux which is something I'm not sure we care about at the moment. That might be true regarding the current in-tree implementation. However, I can almost guarantee that someone wanting to write a web GUI on top of standalone vdsm would want a REST API to talk to. But libvdsm makes this use case of no concern to the core vdsm developers. I do think that having C supportability in our API is a good idea, but the current API should not be used as the base. Let's _start_ with a schema document that describes today's API and then clean it up. I think that will work better than starting from scratch. Once my schema is written I will post it and we can 'patch' it as a community until we arrive at a 1.0 version we are all happy with. +1 Ok. Redoubling my efforts to get this done. Describing the output of list(True) takes awhile :) ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Adam Litke a...@us.ibm.com IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
On 2012-7-12 20:41, Adam Litke wrote: On Thu, Jul 12, 2012 at 08:11:17AM +0800, Shu Ming wrote: Basically, my understanding is that we can generate two versions of libvdsm from the schema file for both the node and the management application. First, the transportation protocols(XMLRPC, REST-API) will depend on libvdsm(node version) to export the APIs to remote management application. Secondly, the management application can use libvdsm(application version ) to emit the remote call to the node. Also, transportation protocols like REST API and XML RPC API can also be generated automatically by the schema file with C, Java, Python bindings. I think this might be a bit too complex of a model. Here's how I see it... The schema generates C/gObject code which can be compiled into libvdsm. We can use the gObject introspection library to automatically generate language bindings for Java, Python, Perl, etc. Do you mean Java, Python, Perl, etc bindings of libvdsm? The libvdsm library talks to vdsmd using a wire protocol that works locally and remotely. This wire protocol is completely hidden from library users. It's an implementation detail that can be changed later if necessary. Today I would recommend that we use xmlrpc. This means that ovirt-engine or another remote program could use libvdsm in the exact same manner as a local program. The library user just needs to call libvdsm.connect(uri). Except libvdsm.connet() or libvdsm.disconnet(), do you expect the engine will not change any of its xml-rpc interfaces now after libvdsm is introduced? I mean engine may not want to change much of its current code, so it expect the least change. Finally, REST and AMQP bridges would be written solely against libvdsm. These So we should write REST, AMQP bridges manually with Java, Python, Perl language on various bindings of libvdsm? bridges are probably not suitable for code generation (but we can revisit that as a separate issue because it's up to the bridge writer to determine the best approach). On 2012-7-12 2:29, Saggi Mizrahi wrote: I'm sorry, but I don't really understand the drawing - Original Message - From: Shu Ming shum...@linux.vnet.ibm.com To: Adam Litke a...@us.ibm.com Cc: vdsm-devel@lists.fedorahosted.org Sent: Wednesday, July 11, 2012 10:24:49 AM Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm Adam, Maybe, I don't fully understand your proposal. Here is my understanding of libvdsm in the picture. Please check the following link for the picture. http://www.ovirt.org/wiki/File:Libvdsm.JPG http://www.ovirt.org/wiki/File:Libvdsm.JPG On 2012-7-9 21:56, Adam Litke wrote: On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote: On 07/06/2012 01:15 AM, Robert Middleswarth wrote: On 07/05/2012 04:45 PM, Adam Litke wrote: On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote: - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: Anthony Liguori anth...@codemonkey.ws, VDSM Project Development vdsm-devel@lists.fedorahosted.org Sent: Thursday, July 5, 2012 2:34:50 PM Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote: The idea of having a supported C API was something I was thinking about doing (But I'd rather use gobject introspection and not schema generation) But the problem is not having a C API is using the current XML RPC API as it's base I want to disect this a bit to find out exactly where there might be agreement and disagreement. C API is a good thing to implement - Agreed. I also want to use gobject introspection but I don't agree that using glib precludes the use of a formalized schema. My proposal is that we write a schema definition and generate the glib C code from that schema. I agree that the _current_ xmlrpc API makes a pretty bad base from which to start a supportable API. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would
[vdsm] Creating a new VM with a template
Hi, I created a VM with from existing template and found that the image of the new VM actually copied the image from the template instead of sharing a base from the template. I am wondering why the new VM doesn't use a new cow image as its image pointing to the base image in template. By that way, all the new VMs will share a common parent image in template and save the space in the storage domain. Is there any other concern not to do this? -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
Basically, my understanding is that we can generate two versions of libvdsm from the schema file for both the node and the management application. First, the transportation protocols(XMLRPC, REST-API) will depend on libvdsm(node version) to export the APIs to remote management application. Secondly, the management application can use libvdsm(application version ) to emit the remote call to the node. Also, transportation protocols like REST API and XML RPC API can also be generated automatically by the schema file with C, Java, Python bindings. On 2012-7-12 2:29, Saggi Mizrahi wrote: I'm sorry, but I don't really understand the drawing - Original Message - From: Shu Ming shum...@linux.vnet.ibm.com To: Adam Litke a...@us.ibm.com Cc: vdsm-devel@lists.fedorahosted.org Sent: Wednesday, July 11, 2012 10:24:49 AM Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm Adam, Maybe, I don't fully understand your proposal. Here is my understanding of libvdsm in the picture. Please check the following link for the picture. http://www.ovirt.org/wiki/File:Libvdsm.JPG http://www.ovirt.org/wiki/File:Libvdsm.JPG On 2012-7-9 21:56, Adam Litke wrote: On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote: On 07/06/2012 01:15 AM, Robert Middleswarth wrote: On 07/05/2012 04:45 PM, Adam Litke wrote: On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote: - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: Anthony Liguori anth...@codemonkey.ws, VDSM Project Development vdsm-devel@lists.fedorahosted.org Sent: Thursday, July 5, 2012 2:34:50 PM Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote: The idea of having a supported C API was something I was thinking about doing (But I'd rather use gobject introspection and not schema generation) But the problem is not having a C API is using the current XML RPC API as it's base I want to disect this a bit to find out exactly where there might be agreement and disagreement. C API is a good thing to implement - Agreed. I also want to use gobject introspection but I don't agree that using glib precludes the use of a formalized schema. My proposal is that we write a schema definition and generate the glib C code from that schema. I agree that the _current_ xmlrpc API makes a pretty bad base from which to start a supportable API. XMLRPC is a perfectly reasonable remote/wire protocol and I think we should continue using it as a base for the next generation API. Using a schema will ensure that the new API is well-structured. There major problems with XML-RPC (and to some extent with REST as well) are high call overhead and no two way communication (push events). Basing on XML-RPC means that we will never be able to solve these issues. I am not sure I am ready to conceed that XML-RPC is too slow for our needs. Can you provide some more detail around this point and possibly suggest an alternative that has even lower overhead without sacrificing the ubiquity and usability of XML-RPC? As far as the two-way communication point, what are the options besides AMQP/ZeroMQ? Aren't these even worse from an overhead perspective than XML-RPC? Regarding two-way communication: you can write AMQP brokers based on the C API and run one on each vdsm host. Assuming the C API supports events, what else would you need? I personally think that using something like AMQP for inter-node communication and engine - node would be optimal. With a rest interface that just send messages though something like AMQP. I would also not dismiss AMQP so soon we want a bug with more than a single listener at engine side (engine, history db, maybe event correlation service). collectd as a means for statistics already supports it as well. I'm for having REST as well, but not sure as main one for a consumer like ovirt engine. I agree that a message bus could be a very useful model of communication between ovirt-engine components and multiple vdsm instances. But the complexities and dependencies of AMQP do not make it suitable for use as a low-level API. AMQP will repel new adopters. Why not establish a libvdsm that is more minimalist and can be easily used by everyone? Then AMQP brokers can be built on top of the stable API with ease. All AMQP should require of the low-level API are standard function calls and an events mechanism. Thanks Robert The current XML-RPC API contains a lot of decencies and inefficiencies and we would like to retire it as soon as we possibly can. Engine would like us to move to a message based API and 3rd parties want something simple like REST so it looks like no one actually wants to use XML-RPC. Not even us. I am proposing that AMQP brokers and REST APIs could be written against the public API. In fact, they need not even live in the vdsm tree
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
. Complex data structures can be broken down into their basic types. These are: int, str, bool, list, dict, typed-dict, enum. I have already started this process and am using Qemu's QAPI schema language. You can see that here [1]. For an example of what that looks like describing the vdsm API see this snippet [2]. 2) Import the parser/generator code from qemu for the above schema. Vdsm will require a few extensions such as typed-dictionaries, tuples, and type aliases. Adapt the generator so that it can produce a libvdsm which provides API language bindings for python, c, and java. 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely auto-generated from the schema and accompanying documentation. This can serve to model how new transports can be written. For example, an AMQP implementation can be implemented entirely outside of the vdsm project if we wished. It only needs to talk to vdsm via libvdsm. Maybe API layer can load API interface definition from this schema. Then we only define the api interface at one place. :) I am not sure now. still thinking. I think the code for vdsm-shell can be auto-generated from the schema at build time. I prefer this over dynamic discovery because it is much simpler.` Easy as 1,2,3 :) [1] http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD [2] http://fpaste.org/rt96/ Probably more than you bargained for when asking for more info... :) -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About code segment in vdsm/storage/image.py
Hi, I am reading the code of merge() function in image.py and get one question about the code. I wonder why we should get the volume parameters from the parent of the destination volume if the parent is existing instead from the destination volume itself? 1101if dstParentUUID != sd.BLANK_UUID: volParams = vols[dstParentUUID].getVolumeParams() else: 1105 volParams = dstVol.getVolumeParams() -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About code segment in vdsm/storage/image.py
On 2012-6-25 15:21, Shu Ming wrote: Hi, I am reading the code of merge() function in image.py and get one question about the code. I wonder why we should get the volume parameters from the parent of the destination volume if the parent is existing instead from the destination volume itself? 1101if dstParentUUID != sd.BLANK_UUID: volParams = vols[dstParentUUID].getVolumeParams() else: 1105 volParams = dstVol.getVolumeParams() In my understanding, the volParams is different in internal volume merge and base volume merge(merging to a top most base without parent). For internal volume merge, the volParams comes from the parent of the destination volume. While for base volume merge, the volParams comes from the destination volume. Is it correct? -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Engine-devel] RFC: Writeup on VDSM-libstoragemgmt integration
On 2012-6-23 20:40, Itamar Heim wrote: On 06/23/2012 03:09 AM, Andy Grover wrote: On 06/22/2012 04:46 PM, Itamar Heim wrote: On 06/23/2012 02:31 AM, Andy Grover wrote: On 06/18/2012 01:15 PM, Saggi Mizrahi wrote: Also, there is no mention on credentials in any part of the process. How does VDSM or the host get access to actually modify the storage array? Who holds the creds for that and how? How does the user set this up? It seems to me more natural to have the oVirt-engine use libstoragemgmt directly to allocate and export a volume on the storage array, and then pass this info to the vdsm on the node creating the vm. This answers Saggi's question about creds -- vdsm never needs array modification creds, it only gets handed the params needed to connect and use the new block device (ip, iqn, chap, lun). Is this usage model made difficult or impossible by the current software architecture? what about live snapshots? I'm not a virt guy, so extreme handwaving: vm X uses luns 1 2 engine - vdsm pause vm X that's pausing the VM. live snapshot isn't supposed to do so. Tough we don't expect to do a pausing operation to the VM when live snaphot is undergoing, the VM should be blocked on the access to specific luns for a while. The blocking time should be very short to avoid the storage IO time out in the VM. engine - libstoragemgmt snapshot luns 1, 2 to luns 3, 4 engine - vdsm snapshot running state of X to Y engine - vdsm unpause vm X if engine had any failure before this step, the VM will remain paused. i.e., we compromised the VM to take a live snapshot. engine - vdsm change Y to use luns 3, 4 ? -- Andy ___ Engine-devel mailing list engine-de...@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Can somebody look why I don't have privilege to access bug #833425
Thanks. It worked. On 2012-6-20 18:37, Itamar Heim wrote: On 06/20/2012 06:44 AM, Shu Ming wrote: Hi, My account was registered by IBM email account shum...@cn.ibm.com. This bug should be VDSM specific and I think we should have enough privilege to access VDSM bugs. fixed now. -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguestfs, virt-install, p2v/v2v tooling. All of these are available, but there isn't an easy way to use this tools through an API. - host management operations - vdsm already does some host level configuration (see networking e.g.) it would be good to think about extending the API to cover other areas of configuration and updates - hardware enumeration - driver level information - storage configuration (we've got a bit of a discussion going around libstoragemgmt here) - performance monitoring/debugging - is the host collecting enough information to do debug/perf analysis - can we support specific configurations of a host that optimize for specific workloads - and can we do this in the API such that third-parties can supply and maintain specific workload configurations All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Ming shum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Can somebody look why I don't have privilege to access bug #833425
Hi, My account was registered by IBM email account shum...@cn.ibm.com. This bug should be VDSM specific and I think we should have enough privilege to access VDSM bugs. -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] RFC: Writeup on VDSM-libstoragemgmt integration
On 2012-5-30 17:38, Deepak C Shetty wrote: Hello All, I have a draft write-up on the VDSM-libstoragemgmt integration. I wanted to run this thru' the mailing list(s) to help tune and crystallize it, before putting it on the ovirt wiki. I have run this once thru Ayal and Tony, so have some of their comments incorporated. I still have few doubts/questions, which I have posted below with lines ending with '?' Comments / Suggestions are welcome appreciated. thanx, deepak [Ccing engine-devel and libstoragemgmt lists as this stuff is relevant to them too] -- 1) Background: VDSM provides high level API for node virtualization management. It acts in response to the requests sent by oVirt Engine, which uses VDSM to do all node virtualization related tasks, including but not limited to storage management. libstoragemgmt aims to provide vendor agnostic API for managing external storage array. It should help system administrators utilizing open source solutions have a way to programmatically manage their storage hardware in a vendor neutral way. It also aims to facilitate management automation, ease of use and take advantage of storage vendor supported features which improve storage performance and space utilization. Home Page: http://sourceforge.net/apps/trac/libstoragemgmt/ libstoragemgmt (LSM) today supports C and python plugins for talking to external storage array using SMI-S as well as native interfaces (eg: netapp plugin ) Plan is to grow the SMI-S interface as needed over time and add more vendor specific plugins for exploiting features not possible via SMI-S or have better alternatives than using SMI-S. For eg: Many of the copy offload features require to use vendor specific commands, which justifies the need for a vendor specific plugin. 2) Goals: 2a) Ability to plugin external storage array into oVirt/VDSM virtualization stack, in a vendor neutral way. 2b) Ability to list features/capabilities and other statistical info of the array 2c) Ability to utilize the storage array offload capabilities from oVirt/VDSM. 3) Details: LSM will sit as a new repository engine in VDSM. VDSM Repository Engine WIP @ http://gerrit.ovirt.org/#change,192 Current plan is to have LSM co-exist with VDSM on the virtualization nodes. Does that mean LSM will be a different daemon process than VDSM? Also, how about the vendor's plugin, another process in the nodes? *Note : 'storage' used below is generic. It can be a file/nfs-export for NAS targets and LUN/logical-drive for SAN targets. VDSM can use LSM and do the following... - Provision storage - Consume storage 3.1) Provisioning Storage using LSM Typically this will be done by a Storage administrator. oVirt/VDSM should provide storage admin the - ability to list the different storage arrays along with their types (NAS/SAN), capabilities, free/used space. - ability to provision storage using any of the array capabilities (eg: thin provisioned lun or new NFS export ) - ability to manage the provisioned storage (eg: resize/delete storage) Once the storage is provisioned by the storage admin, VDSM will have to refresh the host(s) for them to be able to see the newly provisioned storage. 3.1.1) Potential flows: Mgmt - vdsm - lsm: create LUN + LUN Mapping / Zoning / whatever is needed to make LUN available to list of hosts passed by mgmt Mgmt - vdsm: getDeviceList (refreshes host and gets list of devices) Repeat above for all relevant hosts (depending on list passed earlier, mostly relevant when extending an existing VG) Mgmt - use LUN in normal flows. 3.1.2) How oVirt Engine will know which LSM to use ? Normally the way this works today is that user can choose the host to use (default today is SPM), however there are a few flows where mgmt will know which host to use: 1. extend storage domain (add LUN to existing VG) - Use SPM and make sure *all* hosts that need access to this SD can see the new LUN 2. attach new LUN to a VM which is pinned to a specific host - use this host 3. attach new LUN to a VM which is not pinned - use a host from the cluster the VM belongs to and make sure all nodes in cluster can see the new LUN So this model depend on the work of removing storage pool? Flows for which there is no clear candidate (Maybe we can use the SPM host itself which is the default ?) 1. create a new disk without attaching it to any VM So the new floating disk should be exported to all nodes and all VMs? 2. create a LUN for a new storage domain 3.2) Consuming storage using LSM Typically this will be done by a virtualization administrator oVirt/VDSM should allow virtualization admin to - Create a new storage domain using the storage on the array. - Be able to specify whether VDSM should use the storage offload capability (default)
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
Saggi, Thanks for writing these down in wiki pages. http://www.ovirt.org/wiki/VDSM_Potential_Features Another possible feature would be make each node in the clusters to be able to maintain the storage domain master data in parallel without a bottle neck in one SPM node. By this way, the storage domain maintain workload will be dispersed into different nodes. On 2012-6-19 5:09, Saggi Mizrahi wrote: Ryan, thanks for commenting. Sadly I feel that your points, though important, are a bit of a digression from the main discussion. Internal architectural changes to VDSM are out of the scope as this should be done on a very tight schedual. Seeing as this is a pretty good list of things that need to be done\discussed in VDSM anyway. I took the liberty of putting them in a wiki page [1] so we don't forget and others can add\comment on the ideas. In any case you can feel free to raise those issues on the list separately. Specifically, 3rd party plugins might be very topical with the undergoing gluster integration work. [1] http://www.ovirt.org/wiki/VDSM_Potential_Features - Original Message - From: Ryan Harperry...@us.ibm.com To: Saggi Mizrahismizr...@redhat.com Cc: VDSM Project Developmentvdsm-devel@lists.fedorahosted.org, Anthony Liguorialigu...@redhat.com Sent: Monday, June 18, 2012 3:43:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Saggi Mizrahismizr...@redhat.com [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguestfs,
[vdsm] __init__ module in VDSM?
Hi,
When I check my vdsm.log, I found those logs. Based on the log
format, the log should be printed by a module with name __init__.
However, after checking the whole files of VDSM workspace, I couldn't
find a __init__.py file with line 1249. My question is where is the
__init__.py file under VDSM?
Thread-807::DEBUG::2012-06-08
00:01:10,730::__init__::1249::Storage.Misc.excCmd::(_log) '/bin/rpm -q
--qf %{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n vdsm' (cwd None)
Thread-807::DEBUG::2012-06-08
00:01:10,752::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: err
= ''; rc = 0
Thread-807::DEBUG::2012-06-08
00:01:10,753::__init__::1249::Storage.Misc.excCmd::(_log) '/bin/rpm -q
--qf %{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n spice-server'
(cwd None)
Thread-807::DEBUG::2012-06-08
00:01:10,773::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: err
= ''; rc = 0
--
Shu Mingshum...@linux.vnet.ibm.com
IBM China Systems and Technology Laboratory
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] A Tool for PEP 8 Patches to Find Code Logic Changes
On 2012-6-7 21:26, Adam Litke wrote:
On Thu, Jun 07, 2012 at 12:03:30PM +0800, Zhou Zheng Sheng wrote:
Hi,
Since there is no coverage report on tests in vdsm, if a PEP 8 patch
passes the tests, we still not sure if there is no mistake in it.
Viewing the diff reports on all the changes consumes a lot of time, and
some small but fatal mistakes(like misspelling variable name) can easily
be ignored by human eyes.
So I have a try on the compiler module of Python. I write a tool named
'pydiff'. pydiff parses two Python scripts into Abstract Syntax Trees.
These data structures can reflect the logic of the code, and pydiff
performs a recursive compare on the trees. Then pydiff reports
differences and the corresponding line numbers. In this way, pydiff
ignores code style changes, and reports only logical changes of the code.
I think this tool can save us a lot of time. After a PEP 8 patch passes
vdsm tests and pydiff, I will get some confidence on the patch and it
probably does not break anything in vdsm.
This is a very nice tool. Thanks for sharing it. I would like to see all
authors of PEP8 patches use this to check their patches for semantic
correctness. This should greatly improve our ability to complete the PEP8
cleanup quickly.
Yes, I agree with you. Also, we should merge this tool into vdsm as a
helper for PEP8 clean work.
Here is a usage example:
test_o.py
def foo(a, b):
pass
if __name__ == '__main__':
A = [1, 2, 3]
print (4, 5, 6), \
over
foo(1, 2)
print 'Hello World'
test_n.py
def foo(a, b):
pass
if __name__ == '__main__':
A = [1,
2, 3]
print (4, 5, 6), over
fooo(
1, 2)
print ('Hello '
'World')
Some differences of the files are just a matter of style. The only
significant difference is the function call foo() is misspelled in
test_n.py.
Run pydiff.py, it will report:
$ python pydiff.py test_*.py
1 difference(s)
first file: test_n.py
second file: test_o.py
((8, 'fooo'), (8, 'foo'))
This report tells us that 'fooo' in line 8 of test_n.py is different
from 'foo' in line 8 of test_o.py.
It can also find insertions or deletions. Here is another simple example:
old.py
print 'Hello 1'
print 'Hello 2'
print 'Hello 3'
print 'Hello 4'
print 'Hello 5'
new.py
print 'Hello 1'
print 'Hello 3'
print 'Hello 4'
print 'Hello 5'
print 'Hello 5'
Run pydiff:
$ pydiff old.py new.py
2 difference(s)
first file: old.py
second file: new.py
((2, Printnl([Const('Hello 2')], None)), (2, None))
((5, None), (5, Printnl([Const('Hello 5')], None)))
Here ((2, Printnl([Const('Hello 2')], None)), (2, None)) means there
is a print statement in line 2 of old.py, but no corresponding statement
in new.py, so we can know the statement is deleted in new.py.
((5, None), (5, Printnl([Const('Hello 5')], None))) means there is a
print statement in line 5 of new.py, but no corresponding statement in
old.py, so we can know the statement is inserted in new.py.
Sometimes the change in code logic is acceptable, for example, change
aDict.has_key(Key) into Key in aDict. pydiff can report a difference
in this case, but it is up to the user to judge whether it's acceptable.
pydiff is just a tool to help you finding these changes.
I hope it can be helpful for PEP 8 patch reviewers. If you find any
bugs, please let me know. The script is in the attachment.
--
Thanks and best regards!
Zhou Zheng Sheng / 周征晟
E-mail: zhshz...@linux.vnet.ibm.com
Telephone: 86-10-82454397
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
--
Shu Mingshum...@linux.vnet.ibm.com
IBM China Systems and Technology Laboratory
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] About the pep8cleaning topic branch
Hi, I am working on pep8cleaning topic branch and include twelve different files in the first batch of topic branch merging into the upstream. And I found the most challenging job was to keep the topic branch synced with the latest master head, especially PEP8 clean can change a file across all the lines of the file. So I am really hoping to narrow the time window of merging the topic branch into master. So I really appreciate that more people can help to review these patches and approve the patches. -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Agenda for tomorrow's call
How about the new repository system latest status? http://gerrit.ovirt.org/#change,192 On 2012-5-21 3:55, Ayal Baron wrote: Hi all, I would like to discuss the following on our upcoming call: - reviewers are missing! - reviewers/verifiers are missing for pep8 patches. I would like to ask a volunteer to aggregate them all in one branch, and get some folks from Red Hat QE to run some sanity test on them. - functional tests: Wenchao Xia's http://gerrit.ovirt.org/#change,4454 and Adam Litke's http://gerrit.ovirt.org/#change,4451 - Saggi's unicode fixes to betterPopen - Stories about negative flows hurt by commit 1676396f18cf5c300d87e181 Change safelease APIs to match SANLock flow - Upcoming oVirt-3.1 release: when to break from master branch? Anyone else has more interesting stuff? We can skip my bullets for a few of yours if we do not have time. Regards, Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] F17's libvirt takes comments into LIBVIRTD_ARGS
On 2012-5-16 18:46, Dan Kenigsberg wrote: On Tue, May 15, 2012 at 04:16:11PM +0800, Shu Ming wrote: On 2012-5-14 7:30, Dan Kenigsberg wrote: On Sun, May 13, 2012 at 11:51:48PM +0800, Shu Ming wrote: Hi, Recently, I found that my host in engine was always in a unassigned state after the host node was installed. After looking into the vdsm.log, it seemed that vdsm failed to call libvirt as an error, libvirtError: Cannot write data: Broken pipe. When I started virsh in the host node at that time, a warning was given WARNING: no socket to connect to and core dumped with virsh net-list. It looks like that no right socket was created for virsh to connect to libvirtd. Any comments about this problem? The followings are my steps in the node: [root@ovirt-node1 ~]# rpm -qa |grep vdsm vdsm-cli-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-python-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-vhostmd-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-4.9.6-0.183.git107644d.fc16.shuming1336622293.x86_64 vdsm-reg-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-debug-plugin-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-hook-faqemu-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch vdsm-bootstrap-4.9.6-0.183.git107644d.fc16.shuming1336622293.noarch [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep libvirt libvirt-daemon-0.9.11-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11-1.fc17.x86_64 libvirt-client-0.9.11-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11-1.fc17.x86_64 libvirt-python-0.9.11-1.fc17.x86_64 [root@ovirt-node1 ~]# virsh net-list WARNING: no socket to connect to Segmentation fault I think that merits a libvirt bug. please attach strace output to bugzilla. [root@ovirt-node1 ~]# [root@ovirt-node1 ~]# ps -ef |grep vdsm root 1299 1 0 23:10 ?00:00:00 /usr/sbin/libvirtd --listen # by vdsm The command line of libvirt process is very odd - the comment that vdsm puts into /etc/sysconfig/libvirtd is somehow taken verbatim. That's bad, and may be related to Fedora 17's systemd services. Try to remove the comment and restart libvirtd to see if this is the case. The comment come from [root@ovirt-node1 ~]# cat /etc/sysconfig/libvirtd: I know that (see my text above). However, in F16 and before, comments have been stripped before being passed to commandline. Have you tested if all is well when the commment is removed? I removed the # by vdsm line from the config file. And restarted the libvirtd and vdsmd service. But no luck to make virsh net-list successful, still got Segmentation fault, while virsh -c qemu:///system -r worked. virsh # net-list Segmentation fault [root@ovirt-node1 ~]# virsh net-list Segmentation fault [root@ovirt-node1 ~]# virsh -c qemu:///system Please enter your authentication name: ^C [root@ovirt-node1 ~]# virsh -c qemu:///system -r Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh net-list Name State Autostart - vdsm-ovirtmgmt active yes virsh Let's see what our friends in libvir-list think. Dan. -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] gerrit problem when reviewing VDSM code
On 2012-5-8 13:24, Itamar Heim wrote: On 05/08/2012 04:50 AM, Shu Ming wrote: Hi, In my reviewing patches in gerrit for VDSM code, I found it was a bit awkward to understand the diffs without much context above or below the diffs. I hope I can read the unchanged lines around the diffs to get more information. The unchanged lines should be in folding state when I don't care about them and can be unfolded when I need more information. What do you think about this? just changed in gerrit the context to show you 'whole file' rather than '10 lines'? Yes. 10 lines don't have much meaning to the people not familiar with the file, but still have an interesting to review the code. More context will help them to review the code without patching the diffs into the original file. -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] gerrit problem when reviewing VDSM code
Hi, In my reviewing patches in gerrit for VDSM code, I found it was a bit awkward to understand the diffs without much context above or below the diffs. I hope I can read the unchanged lines around the diffs to get more information. The unchanged lines should be in folding state when I don't care about them and can be unfolded when I need more information. What do you think about this? -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] vds_bootstrap.py 's residency
Hi, I am checking the VDSM and ovirt-engine workspace for vds_bootstrap.py file. It was found that vds_bootstrap.py file was in VDSM workspace and was packaged into vdsm-bootstrap rpm package. Also, it was found that in the host installation process, host node will try to get the vds_bootstrap.py from the engine server. But vds_bootstrap.py is not included in any engine packages. Does that mean we should install vdsm-bootstrap package into engine server? Why not package this file into engine packages instead of vdsm packages? -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Introducing a validation test package to vdsm
One more comment about the test package version. Most likely, the package version will be the same version as the VDSM package version. The rule we need to consider is: Should we keep the back compatibility with the VDSM files? Say allow newer version test package running on older VDSM files? Or just allow run the same version of test package and VDSM package.One more comment about the test package version. Most likely, the package version will be the same version as the VDSM package version. The rule we need to consider is: Should we keep the back compatibility with the VDSM files? Say allow newer version test package running on older VDSM files? Or just allow run the same version of test package and VDSM package. On 2012-4-26 21:57, Adam Litke wrote: On Thu, Apr 26, 2012 at 05:24:29PM +0800, wenchao xia wrote: Hello, vdsm now have UT suits for developer, but sometimes building and installation machine is not the same one, and additional check is need which is ignored at building time, so I think some test cases should be also run on target machine to check potential errors, Then I want to introduce a sub package as VT suits. Purpose: UT: for developers, more likely a white box, running on building environment. VT: for user and deployment, more likely a black box, running on product or testing environment, all known issue should be covered. Supposed approach: 1 modify building system to generate package: vdsm-VT.rpm. I would prefer the package name 'vdsm-test.rpm' and this package should include unit tests and verification tests. 2 install as an option, after install, user type vdsm-VT would make the test begin. The test runner should be able to run the full suite of unit tests and verification tests (with an option to run only unit tests or only verification tests). This can be the same program that we use in the build environment except that it will set the PYTHONPATH differently to target the installed files. Planned details: 1 Going to place cases in vdsm project in ./tests/VT. 2 On installation will move some useful UT cases into VT. If you follow my approach above, you would simply package the whole tests/ directory and no copying would be necessary. 3 use same framework UT used. 4 two sub dir in test/VT: user_case_test;general_test. What is the difference between these two types of tests? It is just a scratch from my mind, so I'd like hear your opinions. Thanks for the idea! Do you have a sample test for the verification test suite? Will it be your pipe deadlock test? -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Installation dependency for VDSM-4.9.6-0.109 package on FC16
Yes, I tried these. It worked well except some gpg check errors which are harmless I think. On 2012-4-23 20:04, Federico Simoncelli wrote: Hi Shu, you can get sanlock and libvirt from fedora 17: # yum --releasever=17 update libvirt # yum --releasever=17 install sanlock And lvm2 from my repository: http://fsimonce.fedorapeople.org/lvm2/fedora-16/x86_64/ -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Installation dependency for VDSM-4.9.6-0.109 package on FC16
Hi, I am trying to upgrade my VDSM to the latest version. But it seems that several things should be upgraded before the new version of VDSM can be installed on fc16. I am wondering if FC16 is the recommended system for this upgrade. Can anyone tell me if the newer version of Fedora(FC17, FC18) are more compatible with the latest VDSM packages? Dependency check failed here. ---l [root@ovirt-node2 ~]# yum install vdsm Loaded plugins: langpacks, presto, refresh-packagekit fedora/metalink | 9.5 kB 00:00 ovirt-engine | 1.3 kB 00:00 updates/metalink | 4.4 kB 00:00 updates | 4.5 kB 00:00 updates/primary_db | 5.9 MB 00:14 updates/group| 1.9 MB 00:04 vdsm-changes | 2.9 kB 00:00 vdsm-changes/primary_db | 5.4 kB 00:00 Setting up Install Process Resolving Dependencies -- Running transaction check --- Package vdsm.x86_64 0:4.9.3.2-0.fc16 will be updated --- Package vdsm.x86_64 0:4.9.6-0.109.git009bc0c.fc16 will be an update -- Processing Dependency: vdsm-python = 4.9.6-0.109.git009bc0c.fc16 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: libvirt-python = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: lvm2 = 2.02.95 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: libvirt = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: sanlock = 2.1 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: ntp for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: sanlock-python for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Running transaction check --- Package ntp.x86_64 0:4.2.6p4-1.fc16 will be installed -- Processing Dependency: ntpdate = 4.2.6p4-1.fc16 for package: ntp-4.2.6p4-1.fc16.x86_64 --- Package sanlock-python.x86_64 0:1.9-8.fc16 will be installed -- Processing Dependency: sanlock-lib = 1.9-8.fc16 for package: sanlock-python-1.9-8.fc16.x86_64 -- Processing Dependency: libsanlock.so.1()(64bit) for package: sanlock-python-1.9-8.fc16.x86_64 --- Package vdsm.x86_64 0:4.9.6-0.109.git009bc0c.fc16 will be an update -- Processing Dependency: libvirt-python = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: lvm2 = 2.02.95 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: libvirt = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: sanlock = 2.1 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 --- Package vdsm-python.noarch 0:4.9.6-0.109.git009bc0c.fc16 will be installed -- Running transaction check --- Package ntpdate.x86_64 0:4.2.6p4-1.fc16 will be installed --- Package sanlock-lib.x86_64 0:1.9-8.fc16 will be installed --- Package vdsm.x86_64 0:4.9.6-0.109.git009bc0c.fc16 will be an update -- Processing Dependency: libvirt-python = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: lvm2 = 2.02.95 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: libvirt = 0.9.10 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Processing Dependency: sanlock = 2.1 for package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 -- Finished Dependency Resolution Error: Package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 (vdsm-changes) Requires: sanlock = 2.1 Available: sanlock-1.8-1.fc16.x86_64 (fedora) sanlock = 1.8-1.fc16 Available: sanlock-1.9-8.fc16.x86_64 (updates) sanlock = 1.9-8.fc16 Error: Package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 (vdsm-changes) Requires: libvirt-python = 0.9.10 Installed: libvirt-python-0.9.6-4.fc16.x86_64 (@updates) libvirt-python = 0.9.6-4.fc16 Available: libvirt-python-0.9.6-2.fc16.x86_64 (fedora) libvirt-python = 0.9.6-2.fc16 Available: libvirt-python-0.9.6-5.fc16.x86_64 (updates) libvirt-python = 0.9.6-5.fc16 Error: Package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 (vdsm-changes) Requires: lvm2 = 2.02.95 Installed: lvm2-2.02.86-5.fc16.x86_64 (@koji-override-0/$releasever) lvm2 = 2.02.86-5.fc16 Available: lvm2-2.02.86-6.fc16.x86_64 (updates) lvm2 = 2.02.86-6.fc16 Error: Package: vdsm-4.9.6-0.109.git009bc0c.fc16.x86_64 (vdsm-changes) Requires: libvirt = 0.9.10 Installed: libvirt-0.9.6-4.fc16.x86_64 (@updates) libvirt = 0.9.6-4.fc16 Available: libvirt-0.9.6-2.fc16.x86_64
Re: [vdsm] VDSM host network configuration
On 2012-2-16 0:05, Lei Li wrote: Hi, We are working on VDSM network REST APIs, to support the functions we need to get the list of configured networks. I found that VDSM network has a function 'listNetworks' in configNetwork.py. It can get and display the current configured network like this: # python configNetwork.py list Networks: ['bridge_one', 'bridge_three', 'bridge_two'] Vlans: [] Nics: ['eth0'] Bondings: [] But there are some problems with it. It can not display the defined networks after host restart, but the created config file are still there(/etc/sysconfig/network-scripts/..). Did I miss anything? Or Is there some way to avoid this? Lei, How about the real network configurations after the host reboot? You can run brctl show and ifconfig all to get those. Also, service log can be checked to know if there is any failure in the boot process Your suggestion and thoughts would be appreciated. Lei ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live Merge and Storage Live Migration
On 2012-2-7 20:23, Federico Simoncelli wrote: - Original Message - From: Shu Mingshum...@linux.vnet.ibm.com To: Federico Simoncellifsimo...@redhat.com Cc: VDSM Project Developmentvdsm-devel@lists.fedorahosted.org Sent: Tuesday, February 7, 2012 4:10:38 AM Subject: Re: [vdsm] Live Merge and Storage Live Migration I am just curious how HSM interact with SPM after HSM get merge request no matter if they are in the same host. At the moment the engine is responsible to poll the merge status on the HSM (and restart it if failed). When it's successfully completed the deleteVolume command must be issued to the SPM. Sorry for confusing. Please let me explain my question again. In the Engine-VDSM Flow of LiveMerge wiki page, the engine will call the merge interface from engine to HSM to request the merge. I was wondering what HSM will do after it get the merge request. Does it call further interfaces in SPM by whatever way to make SPM to commit the real merge operation or just complete the operation by it self? If it does, how SPM will tell HSM the status of the merge operation?synchronizedly or asnchronizedly? In the future we might consider the mailbox as mean of communication between the HSM and the SPM, even though it's now enabled only for block storage domains (and if the irs.vol_extend_policy is ON). Still the entry point would be the deleteVolume call on the SPM. Does it mean that deleteVolume call will be from HSM to SPM directly instead of from engine? -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Live Merge and Storage Live Migration
I am just curious how HSM interact with SPM after HSM get merge request no matter if they are in the same host. On 2012-2-7 4:34, Federico Simoncelli wrote: During my endless journey back from FOSDEM (24 hours stuck in airports, taxi and trains because of one of the largest blizzards in Italy ever), I've been briefly working on the design for both live merge and storage live migration: http://www.ovirt.org/wiki/Features/Design/LiveMerge http://www.ovirt.org/wiki/Features/Design/StorageLiveMigration They're far for being complete but I'm not able to do anything better now. Please review if you're interested, the general idea should be correct even though there are several unfinished parts. -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] oVirt Live Snapshots
Can someone explain what is DB in this wiki page? See, Live snapshots operation extend regular snapshots as follow: * Create a locked snapshot in DB On 2012-1-30 19:00, Federico Simoncelli wrote: Hi, oVirt, and more specifically VDSM, is currently implementing the live snapshot feature using the API/commands provided by libvirt and qemu. It would be great if you could review the design and the current open issues at: http://ovirt.org/wiki/Live_Snapshots Thank you, -- Shu Mingshum...@linux.vnet.ibm.com IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
