37% is a very low load. Reading disk records takes much longer than parsing the records out of a disk record. With variable record size and moderately poor hashing, overflow is inevitable. So, do you want 80,000 extra groups, or 20,000 overflow buffers? I would go with the smaller number. But for the love of Knuth, do not set your split.load to 90% unless you have a perfectly hashed file with uniformly sized records.
-----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin Sent: Tuesday, July 03, 2012 2:38 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] RESIZE - dynamic files This is why I'm confused.. Is the goal here to reduce 'overflow' or to keep the 'Total size' of the disk down? If the goal is to keep the total disk size down then it would appear you would want your actual load % a lot higher than 37%.. and then ignore 'some' of the overflow.. Chris > But the total size of your file is up 60%. Reading in 60% more records in a > full select of the file is going to be much slower than a few more overflows. > > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin > Sent: Tuesday, July 03, 2012 2:15 PM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] RESIZE - dynamic files > > > Dan, > > I changed the MINIMUM.MODULUS to the value of 200003 as you suggested and my > Actual Load has really gone down (as well as overflow). See below for the > results: > > File name .................. GENACCTRN_POSTED > Pathname ................... GENACCTRN_POSTED > File type .................. DYNAMIC > File style and revision .... 32BIT Revision 12 > Hashing Algorithm .......... GENERAL > No. of groups (modulus) .... 200003 current ( minimum 200003, 5263 empty, > 3957 overflowed, 207 badly ) > Number of records .......... 1290469 > Large record size .......... 3267 bytes > Number of large records .... 180 > Group size ................. 4096 bytes > Load factors ............... 90% (split), 50% (merge) and 37% (actual) > Total size ................. 836235264 bytes > Total size of record data .. 287394719 bytes > Total size of record IDs ... 21508521 bytes > Unused space ............... 527323832 bytes > Total space for records .... 836227072 bytes > > My overflow is now @ 2% > My Load is @ 37% (actual) > > granted my empty groups are now up to almost 3% but I hope that won't be a > big factor. How does this look? > > Chris _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
