When you have a file whose current mod exceeds the base mod, a high percentage of writes will involve splitting & a small re-org. This is enormously expensive. You have to lock the existing data, read it, go grab an available chunk of disk, parse & divide the data according to the algorithm, write both frames out, unlock them, and then read, lock, update, and unlock the header. The update lock of the header occurs earlier in the process than this, so you're holding the lock for a few disk accesses, creating a bottleneck.
The only time I like dynamic files is when I find myself tempted to give a file a seperation of 32 or more. Even then, because I deal with a lot of systems that depend on others to maintain them, I take a look at a GROUP.STAT.DETAIL to see if there's just a few massively oversized items skewing the results, and try to determine if they're heavily used. If FILE.USAGE shows a lot of oversize reads & writes, dynamic might -MIGHT- be a better choice, although I've heard from a usually authoritative source that static files don't do as bad a job as I imagined at storing & retrieving oversized items.
Fact is, I just don't like 'em. It's empirical, not rational. A few times, I've done nothing else to a slow process than convert a dynamic to a static & see an immediate increase in throughput.
Admittedly, one big problem with them is that they have been erroneously presented as an alternative to tuning files, so I'm always running into ones that have gone to seed. I should be more libertarian about it, but repeated encounters with poor maintenance habits have brought out the inner autocrat.
"Our greatest duty in this life is to help others. And please, if you can't help them, could you at least not hurt them?" - H.H. the Dalai Lama
"When buying & selling are controlled by legislation, the first thing to be bought & sold are the legislators" - P.J. O'Rourke
Dan Fitzgerald
From: "Stevenson, Charles" <[EMAIL PROTECTED]> Reply-To: [email protected] To: <[email protected]> Subject: RE: [U2] Universe/Linux Performance Issue Date: Mon, 14 Feb 2005 17:41:38 -0500
-----Original Message-----
I may soon become a fan of static files if this kind of thing keeps up. Here is the output that you are referring to.
Dynamic Files: Slot # Inode Device Ref Count Htype Split Merge Curmod Basemod Largerec Filesp Selects Nextsplit 0 1900545 59425 1 20 80 50 2 2 1628 2344 0 1 1 1835009 59425 110 20 80 50 1943 1024 1628 3223064 0 920
[snip]
Based on the above it looks like a fair amount of file sizing is in order.
Anthony
--------------------------
For starters, I disagree with Dan, whom I respect greatly, about dynamic files. He admits to hating them, I'm partial to them, but I do not use them indiscriminately. (in MY opinion; Dan would say I overuse them.) That said:
Curmod being different from Basemod (analyze.shm -d columns) does not imply needing a resize. The Basemod is just the next power of 2 below the current modulus. It is used by the split/merge algorithm to figure out what group to split or merge next. If Basemod is different from Curmod, that simply means that the current modulus is not a power of 2.
If you do resize a dynamic file, you should think about the minimum.modulus parameter. For example, resizing them with the minimum.modulus set to what the current modulus is would pre-allocate enough _contiguous_ space to house the current data in DATA.30. There is no way for resize to pre-allocate contiguous space in OVER.30. If you let minimum.modulus default to 1, then you will get repeated splits as the file grows, and that space could be all over the disk. In a modern RAID environment, I'm not so sure that matters.
Another approach would be to use a unix cp or mv, or dos copy. I *think* (someone correct me, please!) that the OS looks for enough contiguous space to hold the entire file when deciding where to put it.
cds ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/
------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/
