Il 05/05/2014 19:16, Jiri Moskovcak ha scritto: > On 05/05/2014 03:19 PM, Nir Soffer wrote: >> ----- Original Message ----- >>> From: "Jiri Moskovcak" <[email protected]> >>> To: "Nir Soffer" <[email protected]> >>> Cc: [email protected], "Federico Simoncelli" <[email protected]>, "Allon >>> Mureinik" <[email protected]>, "Greg >>> Padgett" <[email protected]> >>> Sent: Monday, May 5, 2014 3:44:21 PM >>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>> >>> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>>> ----- Original Message ----- >>>>> From: "Jiri Moskovcak" <[email protected]> >>>>> To: "Nir Soffer" <[email protected]> >>>>> Cc: [email protected], "Federico Simoncelli" <[email protected]>, "Allon >>>>> Mureinik" <[email protected]>, "Greg >>>>> Padgett" <[email protected]> >>>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>> >>>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>>> ----- Original Message ----- >>>>>>> From: "Jiri Moskovcak" <[email protected]> >>>>>>> To: "Nir Soffer" <[email protected]> >>>>>>> Cc: [email protected] >>>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>> >>>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>>> ----- Original Message ----- >>>>>>>>> From: "Jiri Moskovcak" <[email protected]> >>>>>>>>> To: [email protected] >>>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>> >>>>>>>>> Greetings vdsm developers! >>>>>>>>> >>>>>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>>>>> ran >>>>>>>>> into a problem with vdms. It seems that when stopped vdsm deactivates >>>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>>> reactivates only specific logical volumes. This is a problem for >>>>>>>>> hosted >>>>>>>>> engine tools as they create logical volumes in the same volume group >>>>>>>>> and >>>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a way >>>>>>>>> to reactivate it, because the services drop the root permissions and >>>>>>>>> are >>>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>>> >>>>>>>> Can you describe what volumes are you creating, and why? >>>>>>> >>>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>>> >>>>>> Do you create these lvs in every vdsm vg? >>>>> >>>>> - only in the first vg created by vdsm while deploying hosted-engine >> >> It seems that the hosted engine has single point of failure - the random >> vg that contains hosted engine data. > > - yes, seems like it, but that's for another discussion > >> >>>>> >>>>>> Is this part of the domain structure >>>>>> used by hosted engine, or it has nothing to do with the storage domain? >>>>> >>>>> - sorry, I don't understand this question. How can I tell if it has >>>>> something to do with the storage domain? It's for storing data about >>>>> hosts set up to run the hosted-engine and data about state of engine and >>>>> the state of VM running the engine. >>>> >>>> Can you tell us exactly what lvs you are creating, and on which vg? >>>> >>>> And how are you creating those lvs - I guess through vdsm? >>>> >>> >>> - no hosted-engine tools do that by calling: >>> >>> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >>> stderr=subprocess.PIPE, >>> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >>> "-n", lv_name, vg_uuid]) >>> .. >> >> How do you ensure that another host is not modifying the same vg in the >> same time? >> >> If you are not ensuring this, you will corrupt this vg sooner or later. >> >> When a storage domain is detached from a host, for example when the host >> is in maintenance mode, lvs on the shared storage may be deleted, >> invalidating >> the devices mapper maps for these devices. If you write to an lv with wrong >> maps, you may be writing to an extent belonging to another lv, corrupting >> that >> lv data, or even worse corrupting the engine vg data. >> > > - when the vdsm wants to move some host to maintenance it contacts the > ha-agent which stops writing data to the storage, but it might be a problem if > the VG is changed while in maintenance, I don't think we handle such situation > >> How do you ensure that the lvs are not deleted while you are using them? > > - obviously we're not handling this, otherwise the vdsm wouldn't be able to > disable those LVs > > - I'm no expert on lvm, so I could use some advice on how to do it, so how > does the vdsm ensures that it's VG is not modified? > >> >>> >>>> The output of lvs command on a host with hosted engine installed will >>>> help us to understand what you are doing, and then we can think more >>>> clearly >>>> what would be the best way to support this in vdsm. >>> >>> The output of lvs: http://fpaste.org/99196/93619139/ >>> >>> HE created these two LVs: >>> ha_agent-hosted-engine.lockspace >>> ha_agent-hosted-engine.metadata >> >> Why do you create these lvs on a vg owned by vdsm? >> > > - I actually don't know this decision has been taken before I started working > on the code > >> If you want total control of these lvs, I suggest that you create your own >> vg and put what ever lvs you like there. >> > > Yes, that would solve the problem with vdsm, I hope Sandro or Martin could > explain why we use the vdsm's VG instead of creating our own...
I don't know why hosted engine metadata are in a different lv, I just create the storage domain using vdsm and then call Martin's library for creating the metadata. On NFS they're inside the storage domain, not on a different storage. > > --Jirka > > >>> >>> --Jirka >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> So far the >>>>>>>>> only suitable solution seems to be to change vdsm to only >>>>>>>>> deactivate/activate it's own LVs. >>>>>>>> >>>>>>>> This sounds reasonable. You can add a list of hosted engine lv names >>>>>>>> and skip these volumes when deactivating vdsm volumes. >>>>>>> >>>>>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it >>>>>>> can just disable only LVs known to it, otherwise we would have to change >>>>>>> the list everytime we add some LV to the group >>>>>> >>>>>> vdsm has a list of special lvs, that needs special treatment. Otherwise, >>>>>> it >>>>>> consider any other lv as owned by vdsm, and will deactivate them when >>>>>> they >>>>>> are >>>>>> not used. >>>>>> >>>>>> I agree that this will create a dependency, but this can also be solved. >>>>>> For example, vdsm can load the list from a file installed by hosted >>>>>> engine, >>>>>> like the typical conf.d directories. >>>>>> >>>>> >>>>> - ok, this is something I actually don't have strong opinion about, for >>>>> me adding a file with a list of LVs or tagging the logical volumes is >>>>> almost the same, I just need a way to tell vdsm which LVs to ignore.. >>>>> >>>>>>> >>>>>>>> >>>>>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>>>>>>> lvs that contains this tag. >>>>>>> >>>>>>> - this sounds good, because if we teach vdsm to ignore LVs with some tag >>>>>>> we can add new LVs in future without changing vdsm. This however applies >>>>>>> also to the solution where vdsm only disables it's own LVs, >>>>>> >>>>>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like >>>>>> this >>>>>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new >>>>>> tag >>>>>> with >>>>>> clear semantic than use some random historic value we have. >>>>>> >>>>>>> so it >>>>>>> depends on vdsm devels which solution they find better. I think the >>>>>>> solution without tags is better, because is simpler and others (like >>>>>>> hosted-engine) can just createlv and don't bother with tags.. >>>>>> >>>>>> I think that a generic tag like OVIRT_IGNORE is an easy solution for >>>>>> everyone. >>>>>> >>>>>> Federico, what do you think? >>>>>> >>>>>> Nir >>>>>> >>>>> >>>>> >>> >>> > -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
