md driver, some info

1999-09-21 Thread Poul-Henning Kamp


The "md" driver I committed today is mostly a proof-of-concept thing
which came out of a chat with Peter Wemm.

It acts like a disk in all aspects, although it will be hard to boot
from it :-)

It will do very simple compression, in that a sector (512bytes)
which is filled with the same character throughout, will not be
allocated as a full sector, instead just the byte value is saved.

In practice this means that making a 10M filesystem doesn't take
10M of ram, until you fill data into it.  The driver will also free
the sectors after you delete a file from the filesystem (ufs only).

# disklabel -r -w md0 auto
# newfs md0c
# vmstat -m | grep "   MD s"
   MD sectors   635   318K318K 21136K  6350 0  512
# mount /dev/md0c /mnt
# cp /kernel /mnt
# vmstat -m | grep "   MD s"
   MD sectors  4178  2089K   2089K 21136K 41780 0  512
# rm /mnt/kernel
# vmstat -m | grep "   MD s"
   MD sectors   636   318K   2120K 21136K 42400 0  512
# umount /mnt
# dd if=/dev/zero of=/dev/rmd0
# vmstat -m | grep "   MD s"
   MD sectors 0 0K   2120K 21136K 42400 0  512
#

Currently the size is arbitrarily set at 2 sectors, just shy
of 10MB.

The driver can be nice-ified in various ways, suggestions and 
patches are welcome.

--
Poul-Henning Kamp FreeBSD coreteam member
[EMAIL PROTECTED]   "Real hackers run -current on their laptop."
FreeBSD -- It will take a long time before progress goes too far!


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: md driver, some info

1999-09-21 Thread Matthew Dillon

:The "md" driver I committed today is mostly a proof-of-concept thing
:which came out of a chat with Peter Wemm.
:
:It acts like a disk in all aspects, although it will be hard to boot
:from it :-)
:
:It will do very simple compression, in that a sector (512bytes)
:which is filled with the same character throughout, will not be
:allocated as a full sector, instead just the byte value is saved.
:
:In practice this means that making a 10M filesystem doesn't take
:10M of ram, until you fill data into it.  The driver will also free
:the sectors after you delete a file from the filesystem (ufs only).
:
:# disklabel -r -w md0 auto
:# newfs md0c
:# vmstat -m | grep "   MD s"
:   MD sectors   635   318K318K 21136K  6350 0  512
:# mount /dev/md0c /mnt
:# cp /kernel /mnt
:# vmstat -m | grep "   MD s"
:   MD sectors  4178  2089K   2089K 21136K 41780 0  512
:# rm /mnt/kernel
:# vmstat -m | grep "   MD s"
:   MD sectors   636   318K   2120K 21136K 42400 0  512
:# umount /mnt
:# dd if=/dev/zero of=/dev/rmd0
:# vmstat -m | grep "   MD s"
:   MD sectors 0 0K   2120K 21136K 42400 0  512
:#

That's very interesting.  You can do the same thing with the VN
device, using swap-backing, except it does not do any compression.

test3:/root# vnconfig -s labels -c -S 32g vn0
test3:/root# pstat -s
Device  1K-blocks UsedAvail Capacity  Type
/dev/od0b 10484480  1048448 0%Interleaved
/dev/da1b 10484480  1048448 0%Interleaved
/dev/da2b 10484480  1048448 0%Interleaved
Total 31453440  3145344 0%

test3:/root# disklabel -r -w vn0 auto
test3:/root# newfs /dev/rvn0c
Warning: Block size restricts cylinders per group to 8.
/dev/rvn0c: 67108864 sectors in 2048 cylinders of 1 tracks, 32768 sectors
32768.0MB in 256 cyl groups (8 c/g, 128.00MB/g, 8128 i/g)
super-block backups (for fsck -b #) at:
 32, 262176, 524320, 786464, 1048608, 1310752, 1572896, 1835040, 2097184,
 2359328, 2621472, 2883616, 3145760, 3407904, 3670048, 3932192, 4194336,
 4456480, 4718624, 4980768, 5242912, 5505056, 5767200, 6029344, 6291488,
...
 62128160, 62390304, 62652448, 62914592, 63176736, 63438880, 63701024,
 63963168, 64225312, 64487456, 64749600, 65011744, 65273888, 65536032,
 65798176, 66060320, 66322464, 66584608, 66846752

test3:/root#  pstat -s
Device  1K-blocks UsedAvail Capacity  Type
/dev/od0b 104844888092   960356 8%Interleaved
/dev/da1b 104844888064   960384 8%Interleaved
/dev/da2b 104844888064   960384 8%Interleaved
Total 3145344   264220  2881124 8%
test3:/root# mount /dev/vn0c /mnt
test3:/root# cp /kernel /mnt
test3:/root# 
test3:/root# pstat -s
Device  1K-blocks UsedAvail Capacity  Type
/dev/od0b 104844888764   959684 8%Interleaved
/dev/da1b 104844888704   959744 8%Interleaved
/dev/da2b 104844888704   959744 8%Interleaved
Total 3145344   266172  2879172 8%

test3:/root# rm /mnt/kernel
test3:/root# pstat -s
Device  1K-blocks UsedAvail Capacity  Type
/dev/od0b 104844888092   960356 8%Interleaved
/dev/da1b 104844888064   960384 8%Interleaved
/dev/da2b 104844888064   960384 8%Interleaved
Total 3145344   264220  2881124 8%

test3:/root# umount /mnt
test3:/root# vnconfig -u vn0
test3:/root# pstat -s
Device  1K-blocks UsedAvail Capacity  Type
/dev/od0b 10484480  1048448 0%Interleaved
/dev/da1b 10484480  1048448 0%Interleaved
/dev/da2b 10484480  1048448 0%Interleaved
Total 31453440  3145344 0%


However, for general use it is recommended that one use the 
'reserve' flag for swap-backed VN filesystems to pre-reserve all
necessary swap in order to maintain performance.  And enabling
softupdates on it once you've newfs'd is also a good idea.

Just for kicks:

test3:/root# vnconfig -s labels -c -S 4t vn0
test3:/root# disklabel -r -w vn0 auto
test3:/root# disklabel vn0
...
bytes/sector: 4096
8 partitions:
#size   offsetfstype   [fsize bsize bps/cpg]
  c: 10737418240unused0 0   # (Cyl.0 - 4194303)
test3:/root# pstat -s
Device  1K-blocks UsedAvail Capacity  Type
Total 31453448  3145336 0%

test3:/root# newfs /dev/rvn0c
preposterous size 0 ---

I find that totally amusing.  I think I'm going to fix newfs, it really
should be able to handle a 4 terrabyte filesystem.

:--
:Poul-Henning Kamp FreeBSD coreteam member
:[EMAIL PROTECTED]   "Real hackers run -current on their laptop."

-Matt
Matthew Dillon 
[EMAIL