On Mon, Mar 11, 2013 at 4:05 PM, Iustin Pop <[email protected]> wrote: > On Mon, Mar 11, 2013 at 03:57:20PM +0100, Helga Velroyen wrote: >> Hi! >> >> >> On Mon, Mar 11, 2013 at 3:34 PM, Helga Velroyen <[email protected]> wrote: >> >> > Hi! >> > >> > On Mon, Mar 11, 2013 at 3:32 PM, Guido Trotter <[email protected]>wrote: >> > >> >> On Mon, Mar 11, 2013 at 7:25 AM, Iustin Pop <[email protected]> wrote: >> >> > On Mon, Mar 11, 2013 at 03:20:36PM +0100, Helga Velroyen wrote: >> >> >> Hi! >> >> >> >> >> >> On Mon, Mar 11, 2013 at 3:18 PM, Iustin Pop <[email protected]> wrote: >> >> >> >> >> >> > On Mon, Mar 11, 2013 at 06:28:26AM -0700, Guido Trotter wrote: >> >> >> > > On Mon, Mar 11, 2013 at 6:17 AM, Helga Velroyen <[email protected] >> >> > >> >> >> > wrote: >> >> >> > > > This patch updates the design doc "Design correct reporting of >> >> storage >> >> >> > > > free space". The modifications were chosen to not conflict with >> >> any >> >> >> > > > future changes of Ganeti regarding storage pools. >> >> >> > > > >> >> >> > > > Signed-off-by: Helga Velroyen <[email protected]> >> >> >> > > > --- >> >> >> > > > doc/design-storagespace.rst | 118 >> >> >> > +++++++++++++++++++++++++++++++------------- >> >> >> > > > 1 file changed, 83 insertions(+), 35 deletions(-) >> >> >> > > > >> >> >> > > > diff --git a/doc/design-storagespace.rst >> >> b/doc/design-storagespace.rst >> >> >> > > > index 919479f..67603e8 100644 >> >> >> > > > --- a/doc/design-storagespace.rst >> >> >> > > > +++ b/doc/design-storagespace.rst >> >> >> > > > @@ -20,21 +20,19 @@ interaction with different storage types. >> >> >> > > > Configuration changes >> >> >> > > > --------------------- >> >> >> > > > >> >> >> > > > -Each storage type will have a new "pools" parameter added (type >> >> list >> >> >> > of >> >> >> > > > -strings). This will be list of vgs for plain and drbd (note >> >> that we >> >> >> > make >> >> >> > > > -no distinction at this level between allowed vgs and metavgs), >> >> the >> >> >> > list >> >> >> > > > -of rados pools for rados, or the storage directory for file and >> >> >> > > > -sharedfile. The parameters already present in the cluster config >> >> >> > object >> >> >> > > > -will be moved to the storage parameters. >> >> >> > > > - >> >> >> > > > -Since currently file and sharedfile only support a single >> >> directory >> >> >> > this >> >> >> > > > -list will be limited to one. In the future if we'll have >> >> support for >> >> >> > > > -more directories, or for per-nodegroup directories this can be >> >> >> > changed. >> >> >> > > > - >> >> >> > > > -Note that these are just "mechanisms" parameters that define >> >> which >> >> >> > > > -storage pools the cluster can use. Further filtering about >> >> what's >> >> >> > > > -allowed can go in the ipolicy, but these changes are not >> >> covered in >> >> >> > this >> >> >> > > > -design doc. >> >> >> > > > +Add a new attribute "enabled_storage_methods" (type: list of >> >> strings) >> >> >> > to the >> >> >> > > > +cluster config which holds the types of storages, for example, >> >> >> > "plain", "drbd", >> >> >> > > > +or "ext". We consider the first one of the list as the default >> >> method. >> >> >> > > > + >> >> >> > > > +For file storage, we'll report the storage space on the file >> >> storage >> >> >> > dir, >> >> >> > > > +which is currently limited to one directory. In the future, if >> >> we'll >> >> >> > have >> >> >> > > > +support for more directories, or for per-nodegroup directories >> >> this >> >> >> > can be >> >> >> > > > +changed. >> >> >> > > > + >> >> >> > > > +Note that the abovementioned enabled_storage_methods are just >> >> >> > "mechanisms" >> >> >> > > > +parameters that define which storage methods the cluster can >> >> use. >> >> >> > Further >> >> >> > > > +filtering about what's allowed can go in the ipolicy, but these >> >> >> > changes are >> >> >> > > > +not covered in this design doc. >> >> >> > > > >> >> >> > > > Since the ipolicy currently has a list of enabled storage >> >> types, we'll >> >> >> > > > use that to decide which storage type is the default, and to >> >> >> > self-select >> >> >> > > > @@ -47,17 +45,23 @@ RPC changes >> >> >> > > > ----------- >> >> >> > > > >> >> >> > > > The noded RPC call that reports node storage space will be >> >> changed to >> >> >> > > > -accept a list of <method>,<key> string tuples. For each of them >> >> it >> >> >> > will >> >> >> > > > -report the free amount of space found on storage <key> as known >> >> by the >> >> >> > > > -requested method. Methods are for example ``lvm``, >> >> ``filesystem``, >> >> >> > > > -``rados``, and the key would be a volume group name in the case >> >> of >> >> >> > lvm, >> >> >> > > > -a directory name for the filesystem and a rados pool name, for >> >> >> > > > -rados_pool. >> >> >> > > > +accept a list of <method>,<key> string tuples. For each of >> >> them, it >> >> >> > will >> >> >> > > > +report the free amount of storage space found on storage <key> >> >> as >> >> >> > known >> >> >> > > > +by the requested storage type method. For example methods are >> >> ``lvm``, >> >> >> > > > +``filesystem``, or ``rados``, and the key would be a volume >> >> group >> >> >> > name, in >> >> >> > > > +the case of lvm, a directory name for the filesystem and a >> >> rados pool >> >> >> > name >> >> >> > > > +for rados storage. >> >> >> > > > + >> >> >> > > > +For now, we will implement only the storage reporting for >> >> non-shared >> >> >> > storage, >> >> >> > > > +that is ``filesystem`` and ``lvm``. For shared storage methods >> >> like >> >> >> > ``rados`` >> >> >> > > > +and ``ext`` we will not implement a free space calculation, >> >> because >> >> >> > it does >> >> >> > > > +not make sense to query each node for the free space of a >> >> commonly >> >> >> > used >> >> >> > > > +storage. >> >> >> > > > >> >> >> > > > Masterd will know (through a constant map) which storage type >> >> uses >> >> >> > which >> >> >> > > > method for storage calculation (i.e. ``plain`` and ``drbd`` use >> >> >> > ``lvm``, >> >> >> > > > -``file`` and ``sharedfile`` use ``filesystem``, etc) and query >> >> the one >> >> >> > > > -needed (or all of the needed ones). >> >> >> > > > +``file`` uses ``filesystem``, etc) and query the one needed (or >> >> all >> >> >> > of the >> >> >> > > > +needed ones). >> >> >> > > > >> >> >> > > > Note that for file and sharedfile the node knows which >> >> directories are >> >> >> > > > allowed and won't allow any other directory to be queried for >> >> security >> >> >> > > > @@ -73,18 +77,40 @@ Ganeti reporting >> >> >> > > > >> >> >> > > > ``gnt-node list`` will by default report information just about >> >> the >> >> >> > > > default storage type. It will be possible to add fields to ask >> >> about >> >> >> > > > -other ones, if they're enabled. >> >> >> > > > - >> >> >> > > > -``gnt-node info`` will report information about all enabled >> >> storage >> >> >> > > > -types, without querying them (just say which ones are supported >> >> >> > > > -according to the cluster configuration). >> >> >> > > > - >> >> >> > > > -``gnt-node list-storage`` will change to report information >> >> about all >> >> >> > > > -available storage pools in each storage type. An extra flag >> >> will be >> >> >> > > > -added to filter by storage pool name (alternatively we can >> >> implement >> >> >> > > > -this by allowing to query by a list of ``type:pool`` string >> >> tuples to >> >> >> > > > -have a more comprehensive filter). >> >> >> > > > - >> >> >> > > > +other ones, if they're enabled. Examples: >> >> >> > > > + >> >> >> > > > +> gnt-node list >> >> >> > > > +Node DTotal DFree MTotal MNode MFree >> >> Pinst Sinst >> >> >> > > > +mynode1 3.6T 3.6T 64.0G 1023M 62.2G >> >> 1 0 >> >> >> > > > +mynode2 3.6T 3.6T 64.0G 1023M 62.0G >> >> 2 1 >> >> >> > > > +mynode3 3.6T 3.6T 64.0G 1023M 62.3G >> >> 0 2 >> >> >> > > > + >> >> >> > > > +> gnt-node list -o dtotal/lvm/myvg,dfree/rados/myrados >> >> >> > > > +Node DTotal (Lvm, myvg) DFree (Rados, myrados) >> >> >> > > > +mynode1 3.6T - >> >> >> > > > +mynode2 3.6T - >> >> >> > > > + >> >> >> > > >> >> >> > > Could you add a note on how one is supposed to find about the >> >> >> > > "myrados" and "myvg" parts, and perhaps we can also have just >> >> >> > > dtotal/lvm that gives the "first/default" lvm. Also with storage >> >> >> > > pools, won't we go directly dtotal/pool_name without specifying >> >> "lvm" >> >> >> > > or "rados" in between? So perhaps we should "just" as an interim >> >> allow >> >> >> > > two levels, or allow three levels in between but go towards having >> >> >> > > two, in the future. >> >> >> > >> >> >> > Wait a second. I'm thoroughly confused now. >> >> >> > >> >> >> > I thought the discussion was that in the current stage we don't allow >> >> >> > custom names, but only one pool (key=type), and in the future >> >> key=type >> >> >> > migrates to key=name (pointing to the definitions)?! >> >> >> > >> >> >> >> >> >> That would drop the possibility to refer to vg and meta_vg seperately, >> >> at >> >> >> least in this interim state. Do we really want that? It is the only >> >> reason >> >> >> to have this three-part-definition as interim solution before storage >> >> pools >> >> >> are implemented. >> >> > >> >> > `metavg' should not be used for actual storage, so I don't see much gain >> >> > from having it available; htools doesn't take it into consideration, and >> >> > in general Ganeti presumes that the administrator has ensured there's >> >> > enough space (i.e. infinite) for it. >> >> > >> >> > So yes, I don't see a problem there. Handling the metavg correctly will >> >> > be a lot of work in any case… >> >> > >> >> >> >> The other reason is that nowadays you can manually specify a vg, >> >> although then "unsupported things" happen, I guess (from allocation, >> >> etc). >> >> >> >> Since to fully support that we will need storage pools, I agree with >> >> this option, and to drop the three/level in favor of two, as long as >> >> we clearly specify that this "completely" supports only one >> >> vg/directory/etc and "manually specified"ones will need to be >> >> reconciled with pools later. >> >> >> >> >> > Okay, I am fine with the two-level solution as well and will make that >> > clear in the design doc. Will send an interdiff soon. >> > >> > Cheers, >> > Helga >> > >> >> Here is the interdiff: >> >> --- >> doc/design-storagespace.rst | 27 ++++++++++++++++----------- >> 1 file changed, 16 insertions(+), 11 deletions(-) >> >> diff --git a/doc/design-storagespace.rst b/doc/design-storagespace.rst >> index 67603e8..6c86d6e 100644 >> --- a/doc/design-storagespace.rst >> +++ b/doc/design-storagespace.rst >> @@ -75,9 +75,9 @@ These calculations will be implemented in the node >> storage system >> Ganeti reporting >> ---------------- >> >> -``gnt-node list`` will by default report information just about the >> -default storage type. It will be possible to add fields to ask about >> -other ones, if they're enabled. Examples: >> +`gnt-node list`` can be queried for the different storage methods, if they >> +are enabled. By default, it will just report information about the default >> +storage method. Examples: >> >> > gnt-node list >> Node DTotal DFree MTotal MNode MFree Pinst Sinst >> @@ -85,18 +85,23 @@ mynode1 3.6T 3.6T 64.0G 1023M >> 62.2G 1 0 >> mynode2 3.6T 3.6T 64.0G 1023M 62.0G 2 1 >> mynode3 3.6T 3.6T 64.0G 1023M 62.3G 0 2 >> >> -> gnt-node list -o dtotal/lvm/myvg,dfree/rados/myrados >> +> gnt-node list -o dtotal/lvm,dfree/rados >> Node DTotal (Lvm, myvg) DFree (Rados, myrados) >> mynode1 3.6T - >> mynode2 3.6T - >> >> -We want to ensure that this design is forward-compatible with respect to >> -the introduction of storage pools. When storage pools are introduced, we >> -will name the default drbd storage pool 'drbd'. It will be possible to >> -rename it. By default, ``gnt-node`` will report on the default storage, >> -but it will be possible to ask about any of them. With the implementation >> -of storage pools, there will be new functionality to ask about what storage >> -pools are available and of what type. >> +Note that for drbd, we only report the space of the vg and only if it was >> not >> +renamed to something different than the default volume group name. With >> this >> +design, there is also no possibility to ask about the meta volume group. We >> +restrict the design here to make the transition to storage pools easier >> (as it >> +is an interim state only). It is the administrator's responsibility to >> ensure >> +that there is enough space for the meta volume group. >> + >> +When storage pools are implemented, we switch from referencing the storage >> +method to referencing the storage pool name. For that, of course, the pool >> +names need to be unique over all storage methods. The default drbd storage >> pool >> +will be called 'drbd' and it will be possible to rename it. There will be >> new >> +functionality to ask about what storage pools are available and of what >> type. > > Since there's no actual "drbd" storage, just LVM under drbd, the concept > of "drbd storage pool" doesn't make much sense. > > I don't know if this is just an accidental oversight or if it points to > a bigger problem behind the storage pools design (it's not a 1:1 mapping > between storage type and storage pool type). >
I believe this is an oversight, as we agreed that the "pool" for drbd would be the one called (by default) "lvm". drbd as you say is just a storage type that uses an lvm-based storage pool (or two, with a different meta device). :) Thanks, Guido
