*About Data Tiering*
Data Tiering is a method to split data between expensive/fast (special
vdev, SSD/NVMe) tier-1 and cheaper/slow vdevs (disks) tier-2 of a hybrid
data pool. Usually you want hot or performance critical data on the fast
tier and older or uncritical data on the second tier.
With ZFS you can select between three tiering methods
*1. Blockbased tiering based on physical data structures.*
This is the default ZFS method when you use a hybrid ZFS pool with
special vdevs. All data land on the normal slower vdevs beside
performance critical small datablocks, metadata or dedup tables. This
improves read/write access to otherwise slow datablocks. Main advantage
is that this method is set and forget. On use cases with many small
volatile files and many users (like a university mailserver) it helps to
improve overall performance as due fast access to metadata.
*2. Tiering based on ZFS small datablock and recsize settings*
If you set the small blocksize property of a ZFS filesystem, all files
with a blocksize smaller/equal this value are stored on the special
vdev, files with a larger blocksize are stored on regular vdevs.
Blocksize of smaller files is reduced dynamically, recsize is the max
blocksize value.
File=100k, small blocksize=128k and recsize=128k: file is always stored
on the special vdevs (blocksize is always < 128k when filesize is < recsize)
File=200k, small blocksize=128k and recsize=128k: file is stored on the
special vdevs (blocksize 128k)
File=300k, small blocksize=128k and recsize=256k: file is stored on the
regular vdevs (blocksize 256k)
For a office filer where most files are small files, set small block
size to 128k and most office documents are on tier-1 while VM or media
files are on tier-2.
This behaviour is very helpful as you can create a filesystem for
hot/performance critical data and another for other/cold/ archival
without sbs set or a larger recsize. Based on recsize you can decide if
only small files or all files are stored on the fast special vdev. If
you change recsize, you can switch behaviour for further writes. This is
the suggested tiering option on ZFS.
*3. Tiering based on single files*
This is the classic tiering approach. You can decide where files are
stored or to move files between fast tier-1 and slow tier-2. This is
not supported by ZFS. You can tier files when you switch recsize setting
below or above the small blocksize value, then rename the file
and copy it back again This results in a move between tier-1 (special
vdev) and tier-2 (regular vdev) under the identical file path.
For a manual tiering set small blocksize and recsize in menu ZFS
Filesystems. Every saved or copied file (not moved) is then stored on
the selected tier.
Smaller files with a dynamically reduced blocksize are always stored on
the special vdev (makes sense as small files are usually slow on access).
Main problem: You can move a file between tiers but any file edit will
store the file due Copy on Write with the then current recsize setting.
Manual tiering is therefor only an option if the default tier
(recsize<=sbs) is the special vdev and you move cold/older files some
time after last edit to tier-2. Hot/active data or small files is then
always on tier-1, larger and older files on tier-2. This can be
automated via jobs.
*Main difference to Arc/L2Arc Caching*
Caching works on read last/read most ZFS datablocks ex 128k not whole files.
A special vdev tier-1 is not a cache but the area where a whole file is
stored to improve read and write.
Main advantage of file tiering
Full control if a larger file is on the fast or slow tier based on size
and last edit time.
Main disdvantage of file/rulebased tiering
Moving data between tiers produces load and cost performance
Files are locked during tiering
More complicated than blockbased tiering
You must care about capacity in tier-1. When full data land on tier-2.
*Conclusion*
Manual tiering on ZFS based on small blocksize and recsize settings can
make sense if you want hot/new data always on a special vdev and move
older/cold files scriptbased after some time past last edit to the
regular vdevs.
Gea
------------------------------------------
illumos: illumos-discuss
Permalink:
https://illumos.topicbox.com/groups/discuss/Ta9815f4d6c901308-Me9acedc81dac4ac86cda28e9
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
