Re: About btrfs qgroup import/export command
On 09.01.2013 11:17, Miao Xie wrote: > Hi, Arne > > On Wed, 19 Dec 2012 12:40:25 +0100, Arne Jansen wrote: >> On 19.12.2012 12:25, Miao Xie wrote: >>> As we know, there is no backup function for qgroup. when the problem >>> occurs, the users must recover qgroup configuration manually, it is not >>> convenient. And besides that, some users might want to import an existed >>> qgroup configuration into a new filesystem. Btrfs does not have such a >>> function,it can only be done manually. >>> >>> So we want to implement btrfs qgroup import/export commands. >>> 1)'btrfs qgroup export' commands will export qgroup tree >>> into a user's specified file.(stdout by default) >>> >>> 2)user may modify the configuration file firstly and then >>> import it into the filesystem.(by 'btrfs qgroup import' command) >>> >>> The file may be formated as the following: >>> >>> Qgroupid is_compressed is_exclusive limited_sizeparent >>> -- >>> 0/10 0 10G1/0 >>> 1/01 1 20G--- >>> >>> If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, >>> else max referenced size. Here 'parent' exclude ancestral qgroups. >>> >>> Is there any comment about this idea? >> >> The configuration only really makes sense in combination with the existing >> subvolumes. Even if the target has subvolumes under the same name, they >> might have different internal IDs. So it might make more sense to address >> the level 0 qgroups by name. > > Good idea. > >> Also it might be misleading to apply a configuration to an existing fs, as >> it currently is not possible get a correct accounting if the fs is not >> empty. Rescan is not yet implemented. > > Rescan will be implemented in the future, so it is not a main problem > to implement 'btrfs qgroup import/export' commands. > >> So instead of just saving and restoring the qgroup config, it might make >> more sense to create a new filesystem including all subvolumes and quota >> config from a config file. >> But, I'm not completely convinced that this is a features that is needed >> frequently. If I want a standard deployment, I simple write a script that >> creates the fs + subvol + quota. > > If users want to config some qgroups(reset the limited size, > modify its ancestral qgroups),i think it is more convenient and flexible > to use import/export commands than write a script. > > > Above all,our qgroup import/export commands will be implemented as follows: > > qgroupid is_compressed is_exclusive limited_size parent full_path > > > And we may specify matching degree when we import the qgroup information. > > <1>strict matching > qgroup(level-0) matches a subvolume/snapshot 's objectid and full path > exactly. > If a qgroup fail to match, the process will exit. > > <2>general matching > It only require qgroup(level-0) to match a subvolume/snapshot 's full > path. > If the corresponding subvolume/snapshot does not exist,skip it. > Otherwise,apply > modifications to the corresponding subvolume/snapshot qgroup. > > <3>weak matching > It only require qgroup(level-0) to match a subvolume/snapshot 's full > path. > If the corresponding subvolume/snapshot does not exist,create the > subvolume > automatically(a tracking qgroup is also created automatically)and then > apply > modifications to the newly created tracking qgroup. > > > How do you think about the above idea? I still have problems imagining a use case for this. In our setup we have lots of subvolumes with a quota configuration that follows some rules, but it won't be possible to just import/export from one machine to the other. So you have to design the tool to your needs. There's another essential tool that's still missing with regard to quota which I'd love to see come to life: Currently the configuration of the tracking qgroups is completely left to the user. This requires a deep understanding how qgroups work from him. It would be great if we could come up with a simple description language where the user just describes what he wants to achieve and the tool calculates the tracking groups itself. It could also contain a templating mechanism that might cover your use case. A description might contain information e.g. which subvols to group, from which subvols the user intends to take snapshots in the future and in which groups those snapshots will be put. My pdf gives some example use cases which should be possible to cover. That's not exactly what you have in mind, but maybe it is possible to cover both needs with one tool. -Arne > > Thanks > Miao > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org >
Re: About btrfs qgroup import/export command
Hi, Arne On Wed, 19 Dec 2012 12:40:25 +0100, Arne Jansen wrote: > On 19.12.2012 12:25, Miao Xie wrote: >> As we know, there is no backup function for qgroup. when the problem >> occurs, the users must recover qgroup configuration manually, it is not >> convenient. And besides that, some users might want to import an existed >> qgroup configuration into a new filesystem. Btrfs does not have such a >> function,it can only be done manually. >> >> So we want to implement btrfs qgroup import/export commands. >> 1)'btrfs qgroup export' commands will export qgroup tree >> into a user's specified file.(stdout by default) >> >> 2)user may modify the configuration file firstly and then >> import it into the filesystem.(by 'btrfs qgroup import' command) >> >> The file may be formated as the following: >> >> Qgroupid is_compressed is_exclusive limited_sizeparent >> -- >> 0/10 0 10G1/0 >> 1/01 1 20G--- >> >> If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, >> else max referenced size. Here 'parent' exclude ancestral qgroups. >> >> Is there any comment about this idea? > > The configuration only really makes sense in combination with the existing > subvolumes. Even if the target has subvolumes under the same name, they > might have different internal IDs. So it might make more sense to address > the level 0 qgroups by name. Good idea. > Also it might be misleading to apply a configuration to an existing fs, as > it currently is not possible get a correct accounting if the fs is not > empty. Rescan is not yet implemented. Rescan will be implemented in the future, so it is not a main problem to implement 'btrfs qgroup import/export' commands. > So instead of just saving and restoring the qgroup config, it might make > more sense to create a new filesystem including all subvolumes and quota > config from a config file. > But, I'm not completely convinced that this is a features that is needed > frequently. If I want a standard deployment, I simple write a script that > creates the fs + subvol + quota. If users want to config some qgroups(reset the limited size, modify its ancestral qgroups),i think it is more convenient and flexible to use import/export commands than write a script. Above all,our qgroup import/export commands will be implemented as follows: qgroupid is_compressed is_exclusive limited_size parent full_path And we may specify matching degree when we import the qgroup information. <1>strict matching qgroup(level-0) matches a subvolume/snapshot 's objectid and full path exactly. If a qgroup fail to match, the process will exit. <2>general matching It only require qgroup(level-0) to match a subvolume/snapshot 's full path. If the corresponding subvolume/snapshot does not exist,skip it. Otherwise,apply modifications to the corresponding subvolume/snapshot qgroup. <3>weak matching It only require qgroup(level-0) to match a subvolume/snapshot 's full path. If the corresponding subvolume/snapshot does not exist,create the subvolume automatically(a tracking qgroup is also created automatically)and then apply modifications to the newly created tracking qgroup. How do you think about the above idea? Thanks Miao -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: About btrfs qgroup import/export command
On Wed, 19 Dec 2012 12:40:25 +0100, Arne Jansen wrote: > On 19.12.2012 12:25, Miao Xie wrote: >> Hi, everyone. >> >> As we know, there is no backup function for qgroup. when the problem >> occurs, the users must recover qgroup configuration manually, it is not >> convenient. And besides that, some users might want to import an existed >> qgroup configuration into a new filesystem. Btrfs does not have such a >> function,it can only be done manually. >> >> So we want to implement btrfs qgroup import/export commands. >> 1)'btrfs qgroup export' commands will export qgroup tree >> into a user's specified file.(stdout by default) >> >> 2)user may modify the configuration file firstly and then >> import it into the filesystem.(by 'btrfs qgroup import' command) >> >> The file may be formated as the following: >> >> Qgroupid is_compressed is_exclusive limited_sizeparent >> -- >> 0/10 0 10G1/0 >> 1/01 1 20G--- >> >> If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, >> else max referenced size. Here 'parent' exclude ancestral qgroups. >> >> Is there any comment about this idea? > > The configuration only really makes sense in combination with the existing > subvolumes. Even if the target has subvolumes under the same name, they > might have different internal IDs. So it might make more sense to address > the level 0 qgroups by name. Yeah. Thanks for your suggest. > Also it might be misleading to apply a configuration to an existing fs, as > it currently is not possible get a correct accounting if the fs is not > empty. Rescan is not yet implemented. > So instead of just saving and restoring the qgroup config, it might make > more sense to create a new filesystem including all subvolumes and quota > config from a config file. > But, I'm not completely convinced that this is a features that is needed > frequently. If I want a standard deployment, I simple write a script that > creates the fs + subvol + quota. But if you set a new value for some groups, you must modify your script at the same time, it is a bit troublesome. Thanks Miao -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: About btrfs qgroup import/export command
On 19.12.2012 12:25, Miao Xie wrote: > Hi, everyone. > > As we know, there is no backup function for qgroup. when the problem > occurs, the users must recover qgroup configuration manually, it is not > convenient. And besides that, some users might want to import an existed > qgroup configuration into a new filesystem. Btrfs does not have such a > function,it can only be done manually. > > So we want to implement btrfs qgroup import/export commands. > 1)'btrfs qgroup export' commands will export qgroup tree > into a user's specified file.(stdout by default) > > 2)user may modify the configuration file firstly and then > import it into the filesystem.(by 'btrfs qgroup import' command) > > The file may be formated as the following: > > Qgroupid is_compressed is_exclusive limited_sizeparent > -- > 0/10 0 10G1/0 > 1/01 1 20G--- > > If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, > else max referenced size. Here 'parent' exclude ancestral qgroups. > > Is there any comment about this idea? The configuration only really makes sense in combination with the existing subvolumes. Even if the target has subvolumes under the same name, they might have different internal IDs. So it might make more sense to address the level 0 qgroups by name. Also it might be misleading to apply a configuration to an existing fs, as it currently is not possible get a correct accounting if the fs is not empty. Rescan is not yet implemented. So instead of just saving and restoring the qgroup config, it might make more sense to create a new filesystem including all subvolumes and quota config from a config file. But, I'm not completely convinced that this is a features that is needed frequently. If I want a standard deployment, I simple write a script that creates the fs + subvol + quota. -Arne > > Thanks > Miao > > > > > > > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
About btrfs qgroup import/export command
Hi, everyone. As we know, there is no backup function for qgroup. when the problem occurs, the users must recover qgroup configuration manually, it is not convenient. And besides that, some users might want to import an existed qgroup configuration into a new filesystem. Btrfs does not have such a function,it can only be done manually. So we want to implement btrfs qgroup import/export commands. 1)'btrfs qgroup export' commands will export qgroup tree into a user's specified file.(stdout by default) 2)user may modify the configuration file firstly and then import it into the filesystem.(by 'btrfs qgroup import' command) The file may be formated as the following: Qgroupid is_compressed is_exclusive limited_sizeparent -- 0/10 0 10G1/0 1/01 1 20G--- If 'is_exclusive' is set, 'limited_size' corresponds to max exlusive size, else max referenced size. Here 'parent' exclude ancestral qgroups. Is there any comment about this idea? Thanks Miao -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
