I set the split load based on what Dan suggested: "I'd take the merge down a little, to maybe 30% or even less, and maybe knock the split up a bit - say, 90% - to cut down on the splitting."
I thought this would cut down on splitting. Is there a certain formula, or way to calculate the split.load? What should my SPLIT.LOAD be around, and how do you come up with that %? Chris > From: r...@lynden.com > To: u2-users@listserver.u2ug.org > Date: Tue, 3 Jul 2012 14:45:28 -0700 > Subject: Re: [U2] RESIZE - dynamic files > > 37% is a very low load. Reading disk records takes much longer than parsing > the records out of a disk record. With variable record size and moderately > poor hashing, overflow is inevitable. So, do you want 80,000 extra groups, > or 20,000 overflow buffers? I would go with the smaller number. But for the > love of Knuth, do not set your split.load to 90% unless you have a perfectly > hashed file with uniformly sized records. > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin > Sent: Tuesday, July 03, 2012 2:38 PM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] RESIZE - dynamic files > > > This is why I'm confused.. Is the goal here to reduce 'overflow' or to > keep the 'Total size' of the disk down? If the goal is to keep the total > disk size down then it would appear > you would want your actual load % a lot higher than 37%.. and then ignore > 'some' of the overflow.. > > Chris > > > > But the total size of your file is up 60%. Reading in 60% more records in > > a full select of the file is going to be much slower than a few more > > overflows. > > > > > > -----Original Message----- > > From: u2-users-boun...@listserver.u2ug.org > > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin > > Sent: Tuesday, July 03, 2012 2:15 PM > > To: u2-users@listserver.u2ug.org > > Subject: Re: [U2] RESIZE - dynamic files > > > > > > Dan, > > > > I changed the MINIMUM.MODULUS to the value of 200003 as you suggested and > > my Actual Load has really gone down (as well as overflow). See below for > > the results: > > > > File name .................. GENACCTRN_POSTED > > Pathname ................... GENACCTRN_POSTED > > File type .................. DYNAMIC > > File style and revision .... 32BIT Revision 12 > > Hashing Algorithm .......... GENERAL > > No. of groups (modulus) .... 200003 current ( minimum 200003, 5263 empty, > > 3957 overflowed, 207 badly ) > > Number of records .......... 1290469 > > Large record size .......... 3267 bytes > > Number of large records .... 180 > > Group size ................. 4096 bytes > > Load factors ............... 90% (split), 50% (merge) and 37% (actual) > > Total size ................. 836235264 bytes > > Total size of record data .. 287394719 bytes > > Total size of record IDs ... 21508521 bytes > > Unused space ............... 527323832 bytes > > Total space for records .... 836227072 bytes > > > > My overflow is now @ 2% > > My Load is @ 37% (actual) > > > > granted my empty groups are now up to almost 3% but I hope that won't be a > > big factor. How does this look? > > > > Chris > > > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
