> From: Bob Friesenhahn [mailto:bfrie...@simple.dallas.tx.us]
> On Thu, 12 Jan 2012, Edward Ned Harvey wrote:
> > Suppose you have a 1G file open, and a snapshot of this file is on disk
> > a previous point in time.
> > for ( i=0 ; i<1trillion ; i++ ) {
> >     seek(random integer in range[0 to 1G]);
> >     write(4k);
> > }
> >
> > Something like this would quickly try to write a bunch of separate and
> > scattered 4k blocks at different offsets within the file.  Every 32 of
> > 4k writes would be write-coalesced into a single 128k on-disk block.
> >
> > Sometime later, you read the whole file sequentially such as cp or tar
> > cat.  The first 4k come from this 128k block...  The next 4k come from
> > another 128k block...  The next 4k come from yet another 128k block...
> > Essentially, the file has become very fragmented and scattered about on
> the
> > physical disk.  Every 4k read results in a random disk seek.
> Are you talking about some other filesystem or are you talking about
> zfs?  Because zfs does not work like that ...

In what way?  I've only described behavior of COW and write coalescing.
Which part are you saying is un-ZFS-like?

Before answering, here, let's do some test work:

Create a new pool, with a single disk, no compression or dedup or anything,
called "junk"

run this script.  All it does is generate some data in a file sequentially,
and then randomly overwrite random pieces of the file in random order,
creating snapshots all along the way...  Many times, until the file has been
completely overwritten many times over.  This should be a fragmentation

Then reboot, to ensure cache is clear.
And see how long it takes to sequentially read the original sequential file,
as compared to the highly fragmented one:
cat /junk/.zfs/snapshot/sequential-before/out.txt | pv > /dev/null ; cat
/junk/.zfs/snapshot/random1399/out.txt | pv > /dev/null

While I'm waiting for this to run, I'll make some predictions:
The file is 2GB (16 Gbit) and the disk reads around 1Gbit/sec, so reading
the initial sequential file should take ~16 sec
After fragmentation, it should be essentially random 4k fragments (32768
bits).  I figure each time the head is able to find useful data, it takes
32us to read the 4kb, followed by 10ms random access time...  disk is doing
useful work 0.3% of the time and wasting 99.7% of the time doing random
seeks.  Should be about 300x longer to read the fragmented file.

... (Ding!) ...  Test is done.  Thank you for patiently waiting during this
time warp.  ;-)

Actual result:  15s and 45s.  So it was 3x longer, not 300x.  Either way it
proves the point - but I want to see results that are at least 100x worse
due to fragmentation, to REALLY drive home the point, that fragmentation

I hypothesize, that the mere 3x performance degradation is because I have
only a single 2G file in a 2T pool, and no other activity, and no other
files.  So all my supposedly randomly distributed data might reside very
close to each other on platter...  The combination of short stroke & read
prefetcher could be doing wonders in this case.  So now I'll repeat that
test, but this time...  I allow the sequential data to be written
sequentially again just like before, but after it starts the random
rewriting, I'll run a couple of separate threads writing and removing other
junk to the pool, so the write coalescing will include other files, spread
more across a larger percentage of the total disk, getting closer to the
worst case random distribution on disk...

(destroy & recreate the pool in between test runs...)

Actual result:  15s and 104s.  So it's only 6.9x performance degradation.
That's the worst I can do without hurting myself.  It proves the point, but
not to the magnitude that I expected.

zfs-discuss mailing list

Reply via email to