Re: [E2FSPROGS, RFC] New mke2fs types parsing

2008-02-21 Thread Andreas Dilger
On Feb 20, 2008  17:20 -0500, Theodore Ts'o wrote:
 There are only three things which mke2fs will do, in my design:

This should all go into the mke2fs man page...

 [fs_types]
   ext3 = {
   features = has_journal
   }
   ext4 = {
   features = extents,flex_bg
   inode_size = 256
   }

Presumably the ext4 feature should also have features = has_journal?
If this is the default for ext4, why would it need to be given for ext3?

We should also add dir_nlink,flexbg while we are in there.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

-
To unsubscribe from this list: send the line unsubscribe linux-ext4 in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [E2FSPROGS, RFC] New mke2fs types parsing

2008-02-20 Thread Eric Sandeen
Theodore Ts'o wrote:
 The following patch is a work in progress, but I'm sending it out so
 folks can take a look at it and comment on the general approach.
 
 What this does is change how mke2fs -T works so it can take a comma
 separated list, so you can do things like this:
 
 mke2fs -T ext4,small,news

Is there some hierarchy of what these options are and how they fit
together?  i.e. small  news might go together (in bizarro world...) but
small,large wouldn't make sense - nor would -T ext3,ext4.  Or, if
somebody stores mail  news on the same fs, nad they say -T mail,news
but the mail  news types have conflicting directives...

how will you define what an acceptable composite of subtypes will be?

Thanks,

-Eric

-
To unsubscribe from this list: send the line unsubscribe linux-ext4 in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [E2FSPROGS, RFC] New mke2fs types parsing

2008-02-20 Thread Theodore Tso
On Wed, Feb 20, 2008 at 12:51:21PM -0600, Eric Sandeen wrote:
 Theodore Ts'o wrote:
  The following patch is a work in progress, but I'm sending it out so
  folks can take a look at it and comment on the general approach.
  
  What this does is change how mke2fs -T works so it can take a comma
  separated list, so you can do things like this:
  
mke2fs -T ext4,small,news
 
 Is there some hierarchy of what these options are and how they fit
 together?  i.e. small  news might go together (in bizarro world...) but
 small,large wouldn't make sense - nor would -T ext3,ext4.  Or, if
 somebody stores mail  news on the same fs, nad they say -T mail,news
 but the mail  news types have conflicting directives...
 
 how will you define what an acceptable composite of subtypes will be?

There are only three things which mke2fs will do, in my design:

#1) If the first type is not one of ext2, ext3, ext4, or
ext4dev, mke2fs will determine a suitable default based on either
argv[0] if it is of the form mke2fs.*, or via defaults.fs_type in
/etc/mke2fs.conf.  If neither of these is available it will default to ext2.

#2) If the second type is not one of small, floppy or default,
mke2fs will create a default type by checking the size of the
to-be-created filesystem.  If it is less than 3mb, it is floppy, if
it is less than 512mb, it is small, otherwise it is default.

#3) Once it has the list of types, i.e., ext3,defaults,squid, or
ext2,floppy,rescue, ext4,defaults,largefile, etc. it uses this as
a search path when mke2fs needs to look up some parameter, such as
base_features, or inode_size, etc.  

So suppose we are looking up the inode_size and the fs_types list is
ext3,defaults,squid.  The possible places where the inode_size
parameter cna be found are: defaults.inode_size,
fs_types.ext3.inode_size, fs_types.defaults.inode_size,
fs_types.squid.inode_size.  Mke2fs will look in the most specific
place (fs_types.squid.inode_size), and then successively more general,
all the way up to defaults.inode_size for the inode_size parameter in
/etc/mke2fs.conf.

(Oh, looks like I got things backwards for the string lookups; oops,
I'll fix that.)

So the basic idea is that it's as free form as possible, and it's all
based on defaults getting overridden in the config file.  So if the
mke2fs.conf file has something like this:

[defaults]
base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr
blocksize = 4096
inode_size = 256
inode_ratio = 16384

[fs_types]
ext3 = {
features = has_journal
}
ext4 = {
features = extents,flex_bg
inode_size = 256
}
small = {
blocksize = 1024
inode_size = 128
inode_ratio = 4096
}
news = {
inode_ratio = 4096
}

And the user runs the command:  /sbin/mkfs.ext4 -T news /dev/hda5

(and let's assume for the same of argument that /dev/hda5 is 400
megabytes)

Then mke2fs will expand the fs_types field to be ext4,small,news.
The user could have specified this explicitly as an argument to the -T
option, but 99% of the time, they won't.

So that means that when we look for the inode_ratio parameter, it will
come form fs_types.news.inode_ratio, and mke2fs will use the value
4096.  For the inode_size, the most specific version will be
fs_types.small.inode_size, or 128.   

In terms of handling features, things are a bit more complicated.  The
design is that we use base_features (first looked in [defaults],
[fs_types].ext4, et. al) to determine the base set of features to
initialize feature set flags.  Next, we look for fs_types.ext4.features, 
fs_types.small.features, fs_types.news.features, and if any of them
exist, they are used to modify the feature set.   Just as with tune2fs, 
the ^ character can be used to turn off a feature.  Finally, the argument 
to -O is used to further modify the feature set.

This are a little bit complicated because I wanted to preserve
backwards compatibility with the existing mke2fs.conf semantics, while
still making it possible to incrementally override portions of the
mke2fs configuration parameters in the simplest but yet most powerful
way possible.

In practice, I suspect most poeple will just say:

   mke2fs -T ext4 /dev/hda5

or
   mkfs.ext4 /dev/hda5

or

   mkfs.ext4 -T news /dev/hda5

Some people might also do:

   mke2fs -T ext4,news /dev/hda5

I doubt many people will do something as complicated as:

   mke2fs -T ext4,largefiles,squid,extra-resize-space /dev/hda5

and have a complicated mke2fs.conf file site on their system --- but
if they really want to do that, they can.

I could also imageine people using this to make it easier to create
different kinds of filesystems for creating test cases or benchmarks, so
/etc/mke2fs.conf could contain things like 

[fs_types]
benchmark_tpc_h = {
raid_stride = 31

Re: [E2FSPROGS, RFC] New mke2fs types parsing

2008-02-20 Thread Eric Sandeen
Theodore Tso wrote:
 On Wed, Feb 20, 2008 at 12:51:21PM -0600, Eric Sandeen wrote:
 Theodore Ts'o wrote:
 The following patch is a work in progress, but I'm sending it out so
 folks can take a look at it and comment on the general approach.

 What this does is change how mke2fs -T works so it can take a comma
 separated list, so you can do things like this:

   mke2fs -T ext4,small,news
 Is there some hierarchy of what these options are and how they fit
 together?  i.e. small  news might go together (in bizarro world...) but
 small,large wouldn't make sense - nor would -T ext3,ext4.  Or, if
 somebody stores mail  news on the same fs, nad they say -T mail,news
 but the mail  news types have conflicting directives...

 how will you define what an acceptable composite of subtypes will be?
 
 There are only three things which mke2fs will do, in my design:

I apologize if I was too quick to ask a dumb question rather than
atually read through the patch, but thank you for the text explanation :)

-Eric
-
To unsubscribe from this list: send the line unsubscribe linux-ext4 in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html