RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
Aha! I see where you're going. I'll give that a go! Thanks to all for your input! Perry -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Anthony Youngman Sent: Monday, December 03, 2007 8:21 AM To: 'u2-users@listserver.u2ug.org' Subject: RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation A second, far bigger adjustment (which will account for a lot of the underestimate), is to divide by 1600, not 2000 or 2048. A split factor of 80 means that your groups will, at maximum, be 80% full. 80% of 2048 is 1640 (near enough). That will give a second-cut (and rather more accurate) estimate 25% larger than the original "divide by 2048" estimate. And 20-25% (depending which way you're going) is quite a big error. Cheers, Wol -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mats Carlid Sent: 03 December 2007 10:08 To: u2-users@listserver.u2ug.org Subject: Re: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation A first adjustment is to allow for the overhead that page and record headers and other administrative information uses e.g. divide by 2000 instead of 2048. Each page has d page header, each record as a record header there will be an 'end of item'-character at the end of each key and each record ( if not there will be pointers and/or lengths instead ) and there will be some unused characters where no record fits at the end of the page. I don't know the sizes of these things but I'd feel lucky if I had a file where they totalled less than 48bytes per page. -- mats Perry Taylor skrev: > That's approximately what I did when calculating how big I needed this > file, although I used the average record size rather than the total > bytes since the majority of the records are less than 75 bytes with a > few large records. > > Let's assume for argument's sake that I took the sum of the record and > ID sizes and divided by 2048. The problem is I came up way short. If I > query that data from the existing table here's what I get... > > >> SUM CLM.RUN.STRIP.PROVDATA EVAL "LEN(@ID)" AS "ID BYTES" FMT "10R" >> > EVAL "LEN(@RECORD)" AS "REC BYTES" FMT "10R" EVAL "LEN(@ID) + > LEN(@RECORD)" AS "BYTES" FMT "10R" > > CLM.RUN.STRIP.PROVDATA ID BYTES.. REC BYTES. BYTES. > > >== == == > TOTALS5035139 17035963 22071102 > > > 719305 records summed. > > >> DIVD 22071102 2048 >> > Decimal 10776 remainder 1854 > > See this is where I get into trouble. 10776 is still a far cry from > 19618. What is also confusing is why ANALYZE.FILE reports such larger > totals for the record and ID bytes than I get from my query. See the > confusion? > > Thanks. > > Perry > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald > Sent: Thursday, November 29, 2007 12:19 PM > To: u2-users@listserver.u2ug.org > Subject: RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation > > This is a very rough cut, but if you divide data+id bytes by 2048, > you'll be > in the ballpark. > > so... > > 46Mb + 5Mb = 51Mb/2048=24902 > > I'd pad that a bit (you don't need tightly packed files), possibly as > much as > 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that > disk > space isn't a constraint as we're dealing with a few Mb. > > If you don't know what the data size will be, my admittedly subjective > formula > would be to divide the anticipated number of records by 10 & go with a > number > in that range, assuming no other issues (like the # of id's is going to > be 40 > Billion & it's a 32-bit file). In this case, you'd end up with an > oversized > file (MIN MOD around 72K) , but you can always adjust that if it's a > problem. > I don't mind oversizing nearly as much as undersizing.> Subject: > [U2][UV] > Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 > 10:13:53 > -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > >> I >> > continue to struggle with accurate calculation of MINIMUM.MODULUS> > values for > dynamic files. Static hashed files do not seem to be such a> challenge > for me. > >>> Here is an example where, once again, I was woefully shy...> > >>> > ---> > File > name .. CLM.RUN.STRIP.PROVDATA> Pathname > ...
RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
A second, far bigger adjustment (which will account for a lot of the underestimate), is to divide by 1600, not 2000 or 2048. A split factor of 80 means that your groups will, at maximum, be 80% full. 80% of 2048 is 1640 (near enough). That will give a second-cut (and rather more accurate) estimate 25% larger than the original "divide by 2048" estimate. And 20-25% (depending which way you're going) is quite a big error. Cheers, Wol -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mats Carlid Sent: 03 December 2007 10:08 To: u2-users@listserver.u2ug.org Subject: Re: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation A first adjustment is to allow for the overhead that page and record headers and other administrative information uses e.g. divide by 2000 instead of 2048. Each page has d page header, each record as a record header there will be an 'end of item'-character at the end of each key and each record ( if not there will be pointers and/or lengths instead ) and there will be some unused characters where no record fits at the end of the page. I don't know the sizes of these things but I'd feel lucky if I had a file where they totalled less than 48bytes per page. -- mats Perry Taylor skrev: > That's approximately what I did when calculating how big I needed this > file, although I used the average record size rather than the total > bytes since the majority of the records are less than 75 bytes with a > few large records. > > Let's assume for argument's sake that I took the sum of the record and > ID sizes and divided by 2048. The problem is I came up way short. If I > query that data from the existing table here's what I get... > > >> SUM CLM.RUN.STRIP.PROVDATA EVAL "LEN(@ID)" AS "ID BYTES" FMT "10R" >> > EVAL "LEN(@RECORD)" AS "REC BYTES" FMT "10R" EVAL "LEN(@ID) + > LEN(@RECORD)" AS "BYTES" FMT "10R" > > CLM.RUN.STRIP.PROVDATA ID BYTES.. REC BYTES. BYTES. > > >== == == > TOTALS5035139 17035963 22071102 > > > 719305 records summed. > > >> DIVD 22071102 2048 >> > Decimal 10776 remainder 1854 > > See this is where I get into trouble. 10776 is still a far cry from > 19618. What is also confusing is why ANALYZE.FILE reports such larger > totals for the record and ID bytes than I get from my query. See the > confusion? > > Thanks. > > Perry > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald > Sent: Thursday, November 29, 2007 12:19 PM > To: u2-users@listserver.u2ug.org > Subject: RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation > > This is a very rough cut, but if you divide data+id bytes by 2048, > you'll be > in the ballpark. > > so... > > 46Mb + 5Mb = 51Mb/2048=24902 > > I'd pad that a bit (you don't need tightly packed files), possibly as > much as > 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that > disk > space isn't a constraint as we're dealing with a few Mb. > > If you don't know what the data size will be, my admittedly subjective > formula > would be to divide the anticipated number of records by 10 & go with a > number > in that range, assuming no other issues (like the # of id's is going to > be 40 > Billion & it's a 32-bit file). In this case, you'd end up with an > oversized > file (MIN MOD around 72K) , but you can always adjust that if it's a > problem. > I don't mind oversizing nearly as much as undersizing.> Subject: > [U2][UV] > Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 > 10:13:53 > -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > >> I >> > continue to struggle with accurate calculation of MINIMUM.MODULUS> > values for > dynamic files. Static hashed files do not seem to be such a> challenge > for me. > >>> Here is an example where, once again, I was woefully shy...> > >>> > ---> > File > name .. CLM.RUN.STRIP.PROVDATA> Pathname > ... > CLM.RUN.STRIP.PROVDATA> File type .. DYNAMIC> Hashing > Algorithm .. GENERAL> No. of groups (modulus) 19618 current > ( > minimum 7867, 0 empty,> 8869 overflowed, 4612 badly> )> Number of > records > .. 719305> Large record size .. 75 bytes> Number of > large > r
Re: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
A first adjustment is to allow for the overhead that page and record headers and other administrative information uses e.g. divide by 2000 instead of 2048. Each page has d page header, each record as a record header there will be an 'end of item'-character at the end of each key and each record ( if not there will be pointers and/or lengths instead ) and there will be some unused characters where no record fits at the end of the page. I don't know the sizes of these things but I'd feel lucky if I had a file where they totalled less than 48bytes per page. -- mats Perry Taylor skrev: > That's approximately what I did when calculating how big I needed this > file, although I used the average record size rather than the total > bytes since the majority of the records are less than 75 bytes with a > few large records. > > Let's assume for argument's sake that I took the sum of the record and > ID sizes and divided by 2048. The problem is I came up way short. If I > query that data from the existing table here's what I get... > > >> SUM CLM.RUN.STRIP.PROVDATA EVAL "LEN(@ID)" AS "ID BYTES" FMT "10R" >> > EVAL "LEN(@RECORD)" AS "REC BYTES" FMT "10R" EVAL "LEN(@ID) + > LEN(@RECORD)" AS "BYTES" FMT "10R" > > CLM.RUN.STRIP.PROVDATA ID BYTES.. REC BYTES. BYTES. > > >== == == > TOTALS5035139 17035963 22071102 > > > 719305 records summed. > > >> DIVD 22071102 2048 >> > Decimal 10776 remainder 1854 > > See this is where I get into trouble. 10776 is still a far cry from > 19618. What is also confusing is why ANALYZE.FILE reports such larger > totals for the record and ID bytes than I get from my query. See the > confusion? > > Thanks. > > Perry > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald > Sent: Thursday, November 29, 2007 12:19 PM > To: u2-users@listserver.u2ug.org > Subject: RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation > > This is a very rough cut, but if you divide data+id bytes by 2048, > you'll be > in the ballpark. > > so... > > 46Mb + 5Mb = 51Mb/2048=24902 > > I'd pad that a bit (you don't need tightly packed files), possibly as > much as > 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that > disk > space isn't a constraint as we're dealing with a few Mb. > > If you don't know what the data size will be, my admittedly subjective > formula > would be to divide the anticipated number of records by 10 & go with a > number > in that range, assuming no other issues (like the # of id's is going to > be 40 > Billion & it's a 32-bit file). In this case, you'd end up with an > oversized > file (MIN MOD around 72K) , but you can always adjust that if it's a > problem. > I don't mind oversizing nearly as much as undersizing.> Subject: > [U2][UV] > Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 > 10:13:53 > -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > >> I >> > continue to struggle with accurate calculation of MINIMUM.MODULUS> > values for > dynamic files. Static hashed files do not seem to be such a> challenge > for me. > >>> Here is an example where, once again, I was woefully shy...> > >>> > ---> > File > name .. CLM.RUN.STRIP.PROVDATA> Pathname > ... > CLM.RUN.STRIP.PROVDATA> File type .. DYNAMIC> Hashing > Algorithm .. GENERAL> No. of groups (modulus) 19618 current > ( > minimum 7867, 0 empty,> 8869 overflowed, 4612 badly> )> Number of > records > .. 719305> Large record size .. 75 bytes> Number of > large > records 6614> Group size . 2048 bytes> Load factors > ... 80% (split), 50% (merge) and 80% (actual)> Total size > . 69978112 bytes> Total size of record data .. 46331730 > bytes> > Total size of record IDs ... 5114218 bytes> Unused space ... > 18528068 bytes> Total space for records 69974016 bytes> > File name > .. CLM.RUN.STRIP.PROVDATA> Number per group ( total of > 19618 > groups> )> Average Minimum Maximum StdDev> Group buffers .. > 1.74 1 > 15 1.19> Records
RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
That's approximately what I did when calculating how big I needed this file, although I used the average record size rather than the total bytes since the majority of the records are less than 75 bytes with a few large records. Let's assume for argument's sake that I took the sum of the record and ID sizes and divided by 2048. The problem is I came up way short. If I query that data from the existing table here's what I get... >SUM CLM.RUN.STRIP.PROVDATA EVAL "LEN(@ID)" AS "ID BYTES" FMT "10R" EVAL "LEN(@RECORD)" AS "REC BYTES" FMT "10R" EVAL "LEN(@ID) + LEN(@RECORD)" AS "BYTES" FMT "10R" CLM.RUN.STRIP.PROVDATA ID BYTES.. REC BYTES. BYTES. == == == TOTALS5035139 17035963 22071102 719305 records summed. >DIVD 22071102 2048 Decimal 10776 remainder 1854 See this is where I get into trouble. 10776 is still a far cry from 19618. What is also confusing is why ANALYZE.FILE reports such larger totals for the record and ID bytes than I get from my query. See the confusion? Thanks. Perry -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald Sent: Thursday, November 29, 2007 12:19 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation This is a very rough cut, but if you divide data+id bytes by 2048, you'll be in the ballpark. so... 46Mb + 5Mb = 51Mb/2048=24902 I'd pad that a bit (you don't need tightly packed files), possibly as much as 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that disk space isn't a constraint as we're dealing with a few Mb. If you don't know what the data size will be, my admittedly subjective formula would be to divide the anticipated number of records by 10 & go with a number in that range, assuming no other issues (like the # of id's is going to be 40 Billion & it's a 32-bit file). In this case, you'd end up with an oversized file (MIN MOD around 72K) , but you can always adjust that if it's a problem. I don't mind oversizing nearly as much as undersizing.> Subject: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 10:13:53 -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > I continue to struggle with accurate calculation of MINIMUM.MODULUS> values for dynamic files. Static hashed files do not seem to be such a> challenge for me. > > Here is an example where, once again, I was woefully shy...> > ---> File name .. CLM.RUN.STRIP.PROVDATA> Pathname ... CLM.RUN.STRIP.PROVDATA> File type .. DYNAMIC> Hashing Algorithm .. GENERAL> No. of groups (modulus) 19618 current ( minimum 7867, 0 empty,> 8869 overflowed, 4612 badly> )> Number of records .. 719305> Large record size .. 75 bytes> Number of large records 6614> Group size . 2048 bytes> Load factors ... 80% (split), 50% (merge) and 80% (actual)> Total size . 69978112 bytes> Total size of record data .. 46331730 bytes> Total size of record IDs ... 5114218 bytes> Unused space ... 18528068 bytes> Total space for records 69974016 bytes> > File name .. CLM.RUN.STRIP.PROVDATA> Number per group ( total of 19618 groups> )> Average Minimum Maximum StdDev> Group buffers .. 1.74 1 15 1.19> Records 36.67 10 68 12.15> Large records .. 0.34 1 4 0.58> Data bytes . 2361.69 350 30359 2270.12> Record ID bytes 260.69 70 491 86.71> Unused bytes ... 944.44 24 2068 650.69> Total bytes 3566.83 2048 30720 0.00> > > Number per record ( total of 719305> records )> Average Minimum Maximum StdDev> Data bytes . 64.41 34 28700 353.25> Record ID bytes 7.11 2 20 1.14> Total bytes 71.52 36 28720 354.13> ---> > I cannot seem to figure out the relationship of the current modulus to> the record sizes/counts/group size factors. When I do the math I always> come up short.> > Can someone offer some suggestions for accurate calculation of> MINIMUM.MODULUS?> > Thanks.> > Perry Taylor> ZirMed, Inc.> > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distr
RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
This is a very rough cut, but if you divide data+id bytes by 2048, you'll be in the ballpark. so... 46Mb + 5Mb = 51Mb/2048=24902 I'd pad that a bit (you don't need tightly packed files), possibly as much as 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that disk space isn't a constraint as we're dealing with a few Mb. If you don't know what the data size will be, my admittedly subjective formula would be to divide the anticipated number of records by 10 & go with a number in that range, assuming no other issues (like the # of id's is going to be 40 Billion & it's a 32-bit file). In this case, you'd end up with an oversized file (MIN MOD around 72K) , but you can always adjust that if it's a problem. I don't mind oversizing nearly as much as undersizing.> Subject: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 10:13:53 -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > I continue to struggle with accurate calculation of MINIMUM.MODULUS> values for dynamic files. Static hashed files do not seem to be such a> challenge for me. > > Here is an example where, once again, I was woefully shy...> > ---> File name .. CLM.RUN.STRIP.PROVDATA> Pathname ... CLM.RUN.STRIP.PROVDATA> File type .. DYNAMIC> Hashing Algorithm .. GENERAL> No. of groups (modulus) 19618 current ( minimum 7867, 0 empty,> 8869 overflowed, 4612 badly> )> Number of records .. 719305> Large record size .. 75 bytes> Number of large records 6614> Group size . 2048 bytes> Load factors ... 80% (split), 50% (merge) and 80% (actual)> Total size . 69978112 bytes> Total size of record data .. 46331730 bytes> Total size of record IDs ... 5114218 bytes> Unused space ... 18528068 bytes> Total space for records 69974016 bytes> > File name .. CLM.RUN.STRIP.PROVDATA> Number per group ( total of 19618 groups> )> Average Minimum Maximum StdDev> Group buffers .. 1.74 1 15 1.19> Records 36.67 10 68 12.15> Large records .. 0.34 1 4 0.58> Data bytes . 2361.69 350 30359 2270.12> Record ID bytes 260.69 70 491 86.71> Unused bytes ... 944.44 24 2068 650.69> Total bytes 3566.83 2048 30720 0.00> > > Number per record ( total of 719305> records )> Average Minimum Maximum StdDev> Data bytes . 64.41 34 28700 353.25> Record ID bytes 7.11 2 20 1.14> Total bytes 71.52 36 28720 354.13> ---> > I cannot seem to figure out the relationship of the current modulus to> the record sizes/counts/group size factors. When I do the math I always> come up short.> > Can someone offer some suggestions for accurate calculation of> MINIMUM.MODULUS?> > Thanks.> > Perry Taylor> ZirMed, Inc.> > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.> ---> u2-users mailing list> u2-users@listserver.u2ug.org> To unsubscribe please visit http://listserver.u2ug.org/ _ Connect and share in new ways with Windows Live. http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
I think part of the problem is, because it says MINIMUM.MODULUS, we think of it as analogous to the modulus of a hashed file. It isn't. It's more of a hint to the system about how big the file is likely to be, and how to avoid wasting effort. So if you're creating a file and intend to load it with a lot of data, you can tell the system "don't bother creating a small file, because it's going to grow immediately. You're best off starting with X" (where X is what it's going to be when you've loaded it). A second use is where a file fluctuates rapidly in size, and you want to tell the system "don't bother shrinking it, because it's going to grow again very soon". Cheers, Wol -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: 29 November 2007 15:39 To: u2-users@listserver.u2ug.org Subject: Re: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation My experience seems to demonstrate that setting the MINIMUM.MODULUS to the size of the data works best, and then needs to be adjusted in much the same way a static file would need to be resized. With that said, however, for files that grow and shrink continuously during a given period, i.e. each month, I found that setting the MINIMUM.MODULUS to roughly 2/3 to 3/4 the largest size the file grew during the month to seem to allow the file to perform at a reasonable rate. This is based on trial and error. In your situation I'm hoping there are some good, hard and fast rules you can glean from this list... and good luck because I'm suspecting there aren't any because of the many and varied applications and environments involved. Karl > I continue to struggle with accurate calculation of MINIMUM.MODULUS > values for dynamic files. Static hashed files do not seem to be such a > challenge for me. > > Here is an example where, once again, I was woefully shy... > > --- > File name .. CLM.RUN.STRIP.PROVDATA > Pathname ... CLM.RUN.STRIP.PROVDATA > File type .. DYNAMIC > Hashing Algorithm .. GENERAL > No. of groups (modulus) 19618 current ( minimum 7867, 0 empty, > 8869 overflowed, 4612 badly > ) > Number of records .. 719305 > Large record size .. 75 bytes > Number of large records 6614 > Group size . 2048 bytes > Load factors ... 80% (split), 50% (merge) and 80% (actual) > Total size . 69978112 bytes > Total size of record data .. 46331730 bytes > Total size of record IDs ... 5114218 bytes > Unused space ... 18528068 bytes > Total space for records 69974016 bytes > > File name .. CLM.RUN.STRIP.PROVDATA >Number per group ( total of 19618 groups > ) >AverageMinimumMaximum StdDev > Group buffers .. 1.74 1 15 1.19 > Records 36.67 10 68 12.15 > Large records .. 0.34 1 4 0.58 > Data bytes . 2361.69350 303592270.12 > Record ID bytes 260.69 70491 86.71 > Unused bytes ...944.44 24 2068 650.69 > Total bytes 3566.83 2048 30720 0.00 > > >Number per record ( total of 719305 > records ) >AverageMinimumMaximum StdDev > Data bytes . 64.41 34 28700 353.25 > Record ID bytes 7.11 2 20 1.14 > Total bytes 71.52 36 28720 354.13 > --- > > I cannot seem to figure out the relationship of the current modulus to > the record sizes/counts/group size factors. When I do the math I always > come up short. > > Can someone offer some suggestions for accurate calculation of > MINIMUM.MODULUS? > > Thanks. > > Perry Taylor > ZirMed, Inc. > > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain confidential > and privileged information. Any unauthorized review, use, disclosure or > distribution is prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will be r
Re: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation
My experience seems to demonstrate that setting the MINIMUM.MODULUS to the size of the data works best, and then needs to be adjusted in much the same way a static file would need to be resized. With that said, however, for files that grow and shrink continuously during a given period, i.e. each month, I found that setting the MINIMUM.MODULUS to roughly 2/3 to 3/4 the largest size the file grew during the month to seem to allow the file to perform at a reasonable rate. This is based on trial and error. In your situation I'm hoping there are some good, hard and fast rules you can glean from this list... and good luck because I'm suspecting there aren't any because of the many and varied applications and environments involved. Karl > I continue to struggle with accurate calculation of MINIMUM.MODULUS > values for dynamic files. Static hashed files do not seem to be such a > challenge for me. > > Here is an example where, once again, I was woefully shy... > > --- > File name .. CLM.RUN.STRIP.PROVDATA > Pathname ... CLM.RUN.STRIP.PROVDATA > File type .. DYNAMIC > Hashing Algorithm .. GENERAL > No. of groups (modulus) 19618 current ( minimum 7867, 0 empty, > 8869 overflowed, 4612 badly > ) > Number of records .. 719305 > Large record size .. 75 bytes > Number of large records 6614 > Group size . 2048 bytes > Load factors ... 80% (split), 50% (merge) and 80% (actual) > Total size . 69978112 bytes > Total size of record data .. 46331730 bytes > Total size of record IDs ... 5114218 bytes > Unused space ... 18528068 bytes > Total space for records 69974016 bytes > > File name .. CLM.RUN.STRIP.PROVDATA >Number per group ( total of 19618 groups > ) >AverageMinimumMaximum StdDev > Group buffers .. 1.74 1 15 1.19 > Records 36.67 10 68 12.15 > Large records .. 0.34 1 4 0.58 > Data bytes . 2361.69350 303592270.12 > Record ID bytes 260.69 70491 86.71 > Unused bytes ...944.44 24 2068 650.69 > Total bytes 3566.83 2048 30720 0.00 > > >Number per record ( total of 719305 > records ) >AverageMinimumMaximum StdDev > Data bytes . 64.41 34 28700 353.25 > Record ID bytes 7.11 2 20 1.14 > Total bytes 71.52 36 28720 354.13 > --- > > I cannot seem to figure out the relationship of the current modulus to > the record sizes/counts/group size factors. When I do the math I always > come up short. > > Can someone offer some suggestions for accurate calculation of > MINIMUM.MODULUS? > > Thanks. > > Perry Taylor > ZirMed, Inc. > > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain confidential > and privileged information. Any unauthorized review, use, disclosure or > distribution is prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will be returned > to the originating party with such advisement noted. If you are not the > intended recipient, please contact the sender by reply e-mail and destroy > all copies of the original message. > --- > u2-users mailing list > u2-users@listserver.u2ug.org > To unsubscribe please visit http://listserver.u2ug.org/ > -- Karl Pearson Director of I.T. ATS Industrial Supply, Inc. [EMAIL PROTECTED] http://www.atsindustrial.com 800-789-9300 x29 Local: 801-978-4429 Fax: 801-972-3888 "To mess up your Linux PC, you have to really work at it; to mess up a microsoft PC you just have to work on it." --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
[U2][UV] Dynamic File MINIMUM.MODULUS Calculation
I continue to struggle with accurate calculation of MINIMUM.MODULUS values for dynamic files. Static hashed files do not seem to be such a challenge for me. Here is an example where, once again, I was woefully shy... --- File name .. CLM.RUN.STRIP.PROVDATA Pathname ... CLM.RUN.STRIP.PROVDATA File type .. DYNAMIC Hashing Algorithm .. GENERAL No. of groups (modulus) 19618 current ( minimum 7867, 0 empty, 8869 overflowed, 4612 badly ) Number of records .. 719305 Large record size .. 75 bytes Number of large records 6614 Group size . 2048 bytes Load factors ... 80% (split), 50% (merge) and 80% (actual) Total size . 69978112 bytes Total size of record data .. 46331730 bytes Total size of record IDs ... 5114218 bytes Unused space ... 18528068 bytes Total space for records 69974016 bytes File name .. CLM.RUN.STRIP.PROVDATA Number per group ( total of 19618 groups ) AverageMinimumMaximum StdDev Group buffers .. 1.74 1 15 1.19 Records 36.67 10 68 12.15 Large records .. 0.34 1 4 0.58 Data bytes . 2361.69350 303592270.12 Record ID bytes 260.69 70491 86.71 Unused bytes ...944.44 24 2068 650.69 Total bytes 3566.83 2048 30720 0.00 Number per record ( total of 719305 records ) AverageMinimumMaximum StdDev Data bytes . 64.41 34 28700 353.25 Record ID bytes 7.11 2 20 1.14 Total bytes 71.52 36 28720 354.13 --- I cannot seem to figure out the relationship of the current modulus to the record sizes/counts/group size factors. When I do the math I always come up short. Can someone offer some suggestions for accurate calculation of MINIMUM.MODULUS? Thanks. Perry Taylor ZirMed, Inc. CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
