The actual load is 70% on your file. The split.load of 90 was set after the 
file was loaded. If you leave it at that value, and add another 100,000 
records, your modulus will not grow, but the number of overflowed groups will. 

Perhaps you need to look at is as "80% not overflowed".  Despite the output, I 
doubt that any of those overflows are that bad. 




-Rick

On Jul 3, 2012, at 1:23 PM, "Chris Austin" <cjausti...@hotmail.com> wrote:

> 
> I guess what I need to know is what's an acceptable % of overflow for a 
> dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
> the calculated min modulus)
> I'm still left with ~ 20% of overflow (see below). Is 20% overflow considered 
> acceptable on average or should I keep tinkering with it to reach a lower 
> overflow %?
> 
> Correct me if I'm wrong but it seems the goal here is to REDUCE the overflow 
> % while not creating too many modulus (groups).
> 
> Chris
> 
> 
> File name ..................   GENACCTRN_POSTED
> Pathname ...................   GENACCTRN_POSTED
> File type ..................   DYNAMIC
> File style and revision ....   32BIT Revision 12
> Hashing Algorithm ..........   GENERAL
> No. of groups (modulus) ....   105715 current ( minimum 103889, 114 empty,
>                                            21092 overflowed, 1452 badly )
> Number of records ..........   1290469
> Large record size ..........   3267 bytes
> Number of large records ....   180
> Group size .................   4096 bytes
> Load factors ...............   90% (split), 50% (merge) and 70% (actual)
> Total size .................   522260480 bytes
> Total size of record data ..   287400239 bytes
> Total size of record IDs ...   21508521 bytes
> Unused space ...............   213343528 bytes
> Total space for records ....   522252288 bytes
> 
>> From: r...@lynden.com
>> To: u2-users@listserver.u2ug.org
>> Date: Tue, 3 Jul 2012 13:10:43 -0700
>> Subject: Re: [U2] RESIZE - dynamic files
>> 
>> The split load is not affecting anything here, since it is more than the 
>> actual load.  What your overflow suggests is that you lower the split.load 
>> value to 70$% or below.  You could go ahead and set the merge.load to an 
>> arbitrarily low number ("1"), and it will probably never do a merge, which 
>> would be the same as specifying a minimum.modulus equal to "as large as it 
>> ever gets".  The exception to this is during file creation & clear.file,  
>> when the minimum.modulus value determines the initial disk allocation.  
>> Short of going to an arbitrarily large minimum.modulus, and a very low 
>> split.load, you are going to have some overflow (unless you have sequential 
>> keys & like sized records).
>> 
>> -Rick
>> 
>> -----Original Message-----
>> From: u2-users-boun...@listserver.u2ug.org 
>> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
>> Sent: Tuesday, July 03, 2012 12:54 PM
>> To: u2-users@listserver.u2ug.org
>> Subject: Re: [U2] RESIZE - dynamic files
>> 
>> 
>> Using the formula below, and changing the split to 90% I get the following:
>> 
>> File name ..................   GENACCTRN_POSTED
>> Pathname ...................   GENACCTRN_POSTED
>> File type ..................   DYNAMIC
>> File style and revision ....   32BIT Revision 12
>> Hashing Algorithm ..........   GENERAL
>> No. of groups (modulus) ....   103889 current ( minimum 103889, 114 empty,
>>                                            22249 overflowed, 1764 badly )
>> Number of records ..........   1290469
>> Large record size ..........   3267 bytes
>> Number of large records ....   180
>> Group size .................   4096 bytes
>> Load factors ...............   90% (split), 50% (merge) and 72% (actual)
>> Total size .................   519921664 bytes
>> Total size of record data ..   287400591 bytes
>> Total size of record IDs ...   21508497 bytes
>> Unused space ...............   211004384 bytes
>> Total space for records ....   519913472 bytes
>> 
>> How does this look in terms of performance?
>> 
>> My Actual load went down 8% as well as some overflow but it looks like my 
>> load % is still high at 72% I'm wondering if I should raise the 
>> MINIMUM.MODULUS even more
>> since I still have a decent amount of overflow and not many large records.
>> 
>> Chris
>> 
>> 
>>> From: r...@lynden.com
>>> To: u2-users@listserver.u2ug.org
>>> Date: Tue, 3 Jul 2012 10:21:16 -0700
>>> Subject: Re: [U2] RESIZE - dynamic files
>>> 
>>> (record + id / 4096 or 2048)
>>> 
>>> You need to factor in overhead & the split factor:   (records + ids) * 1.1 
>>> * 1.25  / 4096    (for 80%)
>>> 
>>> If you use a 20% merge factor and a 80% split factor, the file will start 
>>> merging unless you delete 60 percent of your records.  If you use 90% split 
>>> factor, you will have more overflowed groups.  These numbers refer to the 
>>> total amount of data in the file, not to any individual group.
>>> 
>>> For records of the size that you have, I do not see any advantage to using 
>>> a larger, 4096, group size. You will end up with twice the number of 
>>> records per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
>>> 
>>> -Rick
>>> 
>>> -----Original Message-----
>>> From: u2-users-boun...@listserver.u2ug.org 
>>> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
>>> Sent: Tuesday, July 03, 2012 9:48 AM
>>> To: u2-users@listserver.u2ug.org
>>> Subject: Re: [U2] RESIZE - dynamic files
>>> 
>>> 
>>> File name ..................   GENACCTRN_POSTED
>>> Pathname ...................   GENACCTRN_POSTED
>>> File type ..................   DYNAMIC
>>> File style and revision ....   32BIT Revision 12
>>> Hashing Algorithm ..........   GENERAL
>>> No. of groups (modulus) ....   92776 current ( minimum 31, 89 empty,
>>>                                            28229 overflowed, 2485 badly )
>>> Number of records ..........   1290469
>>> Large record size ..........   3267 bytes
>>> Number of large records ....   180
>>> Group size .................   4096 bytes
>>> Load factors ...............   80% (split), 50% (merge) and 80% (actual)
>>> Total size .................   500600832 bytes
>>> Total size of record data ..   287035391 bytes
>>> Total size of record IDs ...   21508449 bytes
>>> Unused space ...............   192048800 bytes
>>> Total space for records ....   500592640 bytes
>>> ----
>>> Using the record above, how would I calculate the following?
>>> 
>>> 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
>>> current number)?
>>> 2) SPLIT - would 90% seem about right?
>>> 3) MERGE - would 20% seem about right?
>>> 4) Large Record Size - does 3276 seem right?
>>> 5) Group Size - should I be using 4096?
>>> 
>>> I'm just a bit confused as to how to set these, I saw the formula to 
>>> calculate the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I 
>>> always get a lower number
>>> than my current modulus..
>>> 
>>> I also saw where it said to simply take your current modulus # and add 
>>> 10-20% and set the MINIMUM.MODULUS based on that..
>>> 
>>> Based on the table above I'm just trying to get an idea of what these 
>>> should be set at.
>>> 
>>> Thanks,
>>> 
>>> Chris
>>> 
>>> 
>>>> From: cjausti...@hotmail.com
>>>> To: u2-users@listserver.u2ug.org
>>>> Date: Tue, 3 Jul 2012 10:28:17 -0500
>>>> Subject: Re: [U2] RESIZE - dynamic files
>>>> 
>>>> 
>>>> Doug,
>>>> 
>>>> When I do the math I come up with a different # (see below):
>>>> 
>>>> File name ..................   TEST_FILE
>>>> Pathname ...................   TEST_FILE
>>>> File type ..................   DYNAMIC
>>>> File style and revision ....   32BIT Revision 12
>>>> Hashing Algorithm ..........   GENERAL
>>>> No. of groups (modulus) ....   82850 current ( minimum 24, 104 empty,
>>>>                                            26225 overflowed, 1441 badly )
>>>> Number of records ..........   1157122
>>>> Large record size ..........   2036 bytes
>>>> Number of large records ....   576
>>>> Group size .................   4096 bytes
>>>> Load factors ...............   80% (split), 50% (merge) and 80% (actual)
>>>> Total size .................   449605632 bytes
>>>> Total size of record data ..   258687736 bytes
>>>> Total size of record IDs ...   19283300 bytes
>>>> Unused space ...............   171626404 bytes
>>>> Total space for records ....   449597440 bytes
>>>> 
>>>> ------
>>>> 258,687,736 bytes - Total size of record data
>>>> 19,283,300 bytes - Total size of record IDs
>>>> ===========
>>>> 277,971,036 bytes (record + id's)
>>>> 
>>>> 277,971,036 / 4,084 = 68,063 bytes (minimum modulus)
>>>> ------
>>>> 
>>>> 68,063 is less than the current modulus of 82,850. Something with this 
>>>> formula doesn't seem right because if I use that formula I always 
>>>> calculate a
>>>> minimum modulus of less than the current modulus.
>>>> 
>>>> Thanks,
>>>> 
>>>> Chris
>>>> 
>>>> 
>>>> 
>>>>> Date: Mon, 2 Jul 2012 16:08:16 -0600
>>>>> From: dave...@gmail.com
>>>>> To: u2-users@listserver.u2ug.org
>>>>> Subject: Re: [U2] RESIZE - dynamic files
>>>>> 
>>>>> Hi Chris:
>>>>> 
>>>>> You cannot get away with not resizing dynamic files in my experience.  The
>>>>> files do not split and merge like we are led to believe.  The separator is
>>>>> not used on dynamic files.  Your Universe file is badly sized.  The math
>>>>> below will get you reasonably file size.
>>>>> 
>>>>> Let's do the math:
>>>>> 
>>>>> 258687736 (Record Size)
>>>>> 192283300 (Key Size)
>>>>> ========
>>>>> 450,971,036 (Data and Key Size)
>>>>> 
>>>>> 4096 (Group Size)
>>>>> - 12   (32 Bit Overhead)
>>>>> ====
>>>>> 4084 Usable Space
>>>>> 
>>>>> 450971036/4084 = Minimum Modulo 110424 (Prime is 110431)
>>>>> 
>>>>> 
>>>>> [ad]
>>>>> I hate doing this math all of the time.  I have a reasonably priced resize
>>>>> program called XLr8Resizer for $99.00 to do this for me.
>>>>> [/ad]
>>>>> 
>>>>> Regards,
>>>>> Doug
>>>>> www.u2logic.com/tools.html
>>>>> "XLr8Resizer for the rest of us"
>>>>> _______________________________________________
>>>>> U2-Users mailing list
>>>>> U2-Users@listserver.u2ug.org
>>>>> http://listserver.u2ug.org/mailman/listinfo/u2-users
>>>> 
>>>> _______________________________________________
>>>> U2-Users mailing list
>>>> U2-Users@listserver.u2ug.org
>>>> http://listserver.u2ug.org/mailman/listinfo/u2-users
>>> 
>>> _______________________________________________
>>> U2-Users mailing list
>>> U2-Users@listserver.u2ug.org
>>> http://listserver.u2ug.org/mailman/listinfo/u2-users
>>> _______________________________________________
>>> U2-Users mailing list
>>> U2-Users@listserver.u2ug.org
>>> http://listserver.u2ug.org/mailman/listinfo/u2-users
>> 
>> _______________________________________________
>> U2-Users mailing list
>> U2-Users@listserver.u2ug.org
>> http://listserver.u2ug.org/mailman/listinfo/u2-users
>> _______________________________________________
>> U2-Users mailing list
>> U2-Users@listserver.u2ug.org
>> http://listserver.u2ug.org/mailman/listinfo/u2-users
> 
> _______________________________________________
> U2-Users mailing list
> U2-Users@listserver.u2ug.org
> http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Reply via email to