You forgot the need to defragment, since someone suggested that my idea of
using the intrinsic look-ahead ability is hampered by hard fragmentation.
-Original Message-
From: Rick Nuckolls
To: 'U2 Users List'
Sent: Fri, Jul 6, 2012 11:20 am
Subject: Re: [U2] RESIZE - dyn
h
F. W. Davison & Company, Inc.
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: 07/06/2012 12:56 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
So is there a performan
BASIC select do you gain anything by reducing overflow?
Chris
> To: u2-users@listserver.u2ug.org
> From: wjhon...@aol.com
> Date: Thu, 5 Jul 2012 20:12:21 -0400
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> A BASIC SELECT cannot use criteria at all.
> It is going to
rs
Sent: Fri, Jul 6, 2012 10:23 am
Subject: Re: [U2] RESIZE - dynamic files
o is there a performance increase in BASIC SELECTS by reducing overflow? Some
eople are saying to reduce disk space to speed up the BASIC SELECT
hile others say to reduce overflow.. I'm a bit confused. All of our prog
ct do you gain anything by reducing overflow?
Chris
> To: u2-users@listserver.u2ug.org
> From: wjhon...@aol.com
> Date: Thu, 5 Jul 2012 20:12:21 -0400
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> A BASIC SELECT cannot use criteria at all.
> It is going to walk thr
On 05/07/12 23:58, Rick Nuckolls wrote:
> Oops, I would of thought that if a file had, say 100,000 bytes, @ 70 percent
> full, there would be 30,000 bytes "empty" or dead. Are you suggesting the
> there would be 70,000 bytes of data and 42,000 bytes of dead space?
Do you mean 100,000 bytes of di
it take to
initially populate the file.
John
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson
Sent: Thursday, July 05, 2012 5:41 PM
To: U2 Users List
Subject: Re: [U2] RESIZE - dynamic files
Chris
rams that
contain the SELECTS.
Chris
> To: u2-users@listserver.u2ug.org
> From: wjhon...@aol.com
> Date: Thu, 5 Jul 2012 20:12:21 -0400
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> A BASIC SELECT cannot use criteria at all.
> It is going to walk through every record in th
...@listserver.u2ug.org] On Behalf Of Wjhonson
Sent: Thursday, July 05, 2012 5:12 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
A BASIC SELECT cannot use criteria at all.
It is going to walk through every record in the file, in order.
And that's the sticky wicket. That
2012 4:43 pm
Subject: Re: [U2] RESIZE - dynamic files
Most disks and disk systems cache huge amounts of information these days, and,
epending on 20 factors or so, one solution will be better than another for a
iven file.
For the wholesale, SELECT F WITH, The fewest disk records will almost
a
s my theory ;)
-Original Message-
From: Rick Nuckolls
To: 'U2 Users List'
Sent: Thu, Jul 5, 2012 2:29 pm
Subject: Re: [U2] RESIZE - dynamic files
Chris,
For the type of use that you described earlier; BASIC selects and reads,
educing overflow will have negligible perfor
r.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists
Sent: Thursday, July 05, 2012 3:24 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
On 05/07/12 16:12, Martin Phillips wrote:
> A file without overflow is not necessarily the best solution
On 05/07/12 14:49, Chris Austin wrote:
>
> Disk space is not a factor, as we are a smaller shop and disk space comes
> cheap. However, one thing I did notice is when I increased the modulus to a
> very large
> number which then increased my disk space to about 3-4x of my record data, my
> SELEC
On 05/07/12 16:12, Martin Phillips wrote:
> A file without overflow is not necessarily the best solution. Winding the
> split load down to 70% means that at least 30% of the file
> is dead space. The implication of this is that the file is larger and will
> take more disk reads to process sequent
Chris,
I can appreciate what you are doing as an academic exercise.
You seem happy how it looks at this moment, where, because you set
"MINIMUM.MODULUS 118681", you ended up with a current load of 63%.
But think about it: as you add records, the load will reach 70%, per
"SPLIT.LOAD 70", th
-Original Message-
From: Rick Nuckolls
To: 'U2 Users List'
Sent: Thu, Jul 5, 2012 2:29 pm
Subject: Re: [U2] RESIZE - dynamic files
Chris,
For the type of use that you described earlier; BASIC selects and reads,
educing overflow will have negligible performance benefit, especi
ustin
Sent: Thursday, July 05, 2012 11:21 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
Rick,
You are correct, I should be using the smaller size (I just haven't changed it
yet). Based on the reading I have done you should
only use the larger group size when the aver
.
To test the performance I need to write a ton of records and then capture the
output and compare the output using timestamps.
Chris
> From: r...@lynden.com
> To: u2-users@listserver.u2ug.org
> Date: Thu, 5 Jul 2012 09:22:02 -0700
> Subject: Re: [U2] RESIZE - dynamic files
>
>
England
> +44 (0)1604-709200
>
>
>
> -Original Message-
> From: u2-users-boun...@listserver.u2ug.org
> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
> Sent: 05 July 2012 15:19
> To: u2-users@listserver.u2ug.org
> Subject: Re: [U2] RESIZE - dynamic f
] On Behalf Of Chris Austin
Sent: 05 July 2012 15:19
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
I was able to drop from 30% overflow to 12% by making 2 changes:
1) changed the split from 80% to 70% (that alone reduce 10% overflow)
2) changed the MINIMUM.MODULUS to
h.john...@datacom.co.nz
> To: u2-users@listserver.u2ug.org
> Date: Wed, 4 Jul 2012 14:05:02 +1200
> Subject: Re: [U2] RESIZE - dynamic files
>
> Doug may have had a key bounce in his input
>
> > Let's do the math:
> >
> > 258687736 (Record Size)
> > 1
tweaked instead. Thanks for all the help so far, I think
this is all starting to make sense.
Chris
> From: ro...@stamina.com.au
> To: u2-users@listserver.u2ug.org
> Date: Wed, 4 Jul 2012 01:36:26 +
> Subject: Re: [U2] RESIZE - dynamic files
>
> I would suggest that then ac
Hi all,
> I wouldn't actually be surprised if QM is like PI.
Drifting away from U2 but the question was asked
The initial implementation of dynamic files in QM was fairly close to that of
PI/open but it was totally reworked long before the
product went onto general release, resulti
On 04/07/12 17:44, Charles Stevenson wrote:
>>SMAT -d (or ANALYZE.SHM -d) see uv/bin/smat[.exe]
> uv/bin/analyze.shm[.exe]
>
> Dynamic Files:
> Slot # Inode Device Ref Count Htype Split Merge Curmod Basemod
> Largerec Filesp Selects Nextsplit
> 0 1285128087 2093077925
On 04/07/12 19:59, Rick Nuckolls wrote:
> I believe PiOpen used a directory with two files in it ‘&$0’ and ‘&$1’
> corresponding to DATA.30 and OVER.30. If the numbers went up from there, I
> think that they corresponded to alternate keys, ie ‘&$2’ and ‘&$3’
> represented DATA.30 and OVER.30 fo
I believe PiOpen used a directory with two files in it ‘&$0’ and ‘&$1’
corresponding to DATA.30 and OVER.30. If the numbers went up from there, I
think that they corresponded to alternate keys, ie ‘&$2’ and ‘&$3’ represented
DATA.30 and OVER.30 for the first alternate key.
I do not think that
On 04/07/12 11:26, Brian Leach wrote:
>> All the other groups effectively get 1 added to their number
> Not exactly.
>
> Sorry to those who already know this, but maybe it's time to go over linear
> hashing in theory ..
>
> Linear hashing was a system devised by Litwin and originally only for
> i
>SMAT -d (or ANALYZE.SHM -d) see uv/bin/smat[.exe]
uv/bin/analyze.shm[.exe]
Dynamic Files:
Slot # Inode Device Ref Count Htype Split Merge Curmod Basemod
Largerec Filesp Selects Nextsplit
0 1285128087 209307792516208050 4001
2048 3267 2782736
This makes it sound as if you might need to search two groups for a record,
which is not correct. If the initial hash is based on the larger modulo, and
the group exists, then the key will be in the higher number group. If the
result of the first hash is larger than the modulus of the of the t
Good explanation, Brian!
To anyone who skipped it because it looked long: read it anyway.
cds
On 7/4/2012 5:26 AM, Brian Leach wrote:
Sorry to those who already know this, but maybe it's time to go over linear
hashing in theory ..
Linear hashing was a system devised by Litwin and originally o
> All the other groups effectively get 1 added to their number
Not exactly.
Sorry to those who already know this, but maybe it's time to go over linear
hashing in theory ..
Linear hashing was a system devised by Litwin and originally only for
in-memory lists. In fact there's some good implementa
Doug may have had a key bounce in his input
> Let's do the math:
>
> 258687736 (Record Size)
> 192283300 (Key Size)
>
The key size is actually 19283300 in Chris' figures
Regarding 68,063 being less than the current modulus of 82,850. I think the
answer may lie in the splitting process
tamina Software
Visage > Better by Design!
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Wednesday, 4 July 2012 7:38 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
Subject: Re: [U2] RESIZE - dynamic files
Chris,
Let's back way up. I take it your original question is a general one,
not specific to one poorly performing problematic file. Is that right?
If so, generally speaking, you just don't get a lot out of fine-tuning
dynamic files.
Tweaking t
Chris,
Let's back way up. I take it your original question is a general one,
not specific to one poorly performing problematic file. Is that right?
If so, generally speaking, you just don't get a lot out of fine-tuning
dynamic files.
Tweaking the default parameters doesn't usually make a w
day, July 03, 2012 2:52 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
I set the split load based on what Dan suggested:
"I'd take the merge down a little, to maybe 30% or even less, and maybe knock
the split up a bit - say, 90% - to cut down on the splitti
o calculate the split.load? What should my SPLIT.LOAD be around,
and how do you come up with that %?
Chris
> From: r...@lynden.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 14:45:28 -0700
> Subject: Re: [U2] RESIZE - dynamic files
>
> 37% is a very low load. Readi
h
F. W. Davison & Company, Inc.
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: 07/03/2012 5:38 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
This is why I
, 2012 2:38 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
This is why I'm confused.. Is the goal here to reduce 'overflow' or to
keep the 'Total size' of the disk down? If the goal is to keep the total
disk size down then it would appear
y
Disks get "bigger" much faster than the rate they get "faster".
So the overflow is the thing to minimize.
-Original Message-
From: Chris Austin
To: u2-users
Sent: Tue, Jul 3, 2012 2:38 pm
Subject: Re: [U2] RESIZE - dynamic files
his is why I'm confused.. I
ailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
> Sent: Tuesday, July 03, 2012 2:15 PM
> To: u2-users@listserver.u2ug.org
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> Dan,
>
> I changed the MINIMUM.MODULUS to the value of 23 as you suggested and
] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 2:15 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
Dan,
I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my
Actual Load has really gone down (as well as overflow). See below for the
Austin
Sent: Tuesday, July 03, 2012 2:15 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
Dan,
I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my
Actual Load has really gone down (as well as overflow). See below for the
results:
File name
m: dangf...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 16:57:34 -0400
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> One rule of thumb is to make sure that you have an average of 10 or less
> items in each group. Going by that, you'd want a minimum
looks like there. That'll give you an average of 6 records per group, not
unreasonably shallow, and it's likely to be a while before you have to resize
again.
> From: cjausti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 15:23:23 -0500
> Subj
ecord IDs ... 21508521 bytes
> Unused space ... 213343528 bytes
> Total space for records 522252288 bytes
>
>> From: r...@lynden.com
>> To: u2-users@listserver.u2ug.org
>> Date: Tue, 3 Jul 2012 13:10:43 -0700
>> Subject: Re: [U2] RESIZE - dynami
s
Total size of record IDs ... 21508521 bytes
Unused space ... 213343528 bytes
Total space for records 522252288 bytes
> From: r...@lynden.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 13:10:43 -0700
> Subject: Re: [U2] RESIZE - dynamic files
>
>
uly 03, 2012 12:54 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
Using the formula below, and changing the split to 90% I get the following:
File name .. GENACCTRN_POSTED
Pathname ... GENACCTRN_POSTED
File type ...
cords.
Chris
From: r...@lynden.com
To: u2-users@listserver.u2ug.org
Date: Tue, 3 Jul 2012 10:21:16 -0700
Subject: Re: [U2] RESIZE - dynamic files
(record + id / 4096 or 2048)
You need to factor in overhead & the split factor: (records + ids) * 1.1 * 1.25 / 4096(for 80%)
If you
large records.
Chris
> From: r...@lynden.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 10:21:16 -0700
> Subject: Re: [U2] RESIZE - dynamic files
>
> (record + id / 4096 or 2048)
>
> You need to factor in overhead & the split factor: (records + ids) *
Doug,
The data is growing over time with this file. Does that mean I should ignore
the formula? Or should I still use a lower MINIMUM.MODULO than the
actual modulo #..
Is the idea to reduce overflow by lowering the split? What is this 'overflow'
referring to?
> > 2) SPLIT - would 90% seem ab
> Date: Tue, 3 Jul 2012 10:21:16 -0700
> Subject: Re: [U2] RESIZE - dynamic files
>
> (record + id / 4096 or 2048)
>
> You need to factor in overhead & the split factor: (records + ids) * 1.1 *
> 1.25 / 4096(for 80%)
>
> If you use a 20% merge factor and a
..
Based on the table above I'm just trying to get an idea of what these should be
set at.
Thanks,
Chris
> From: cjausti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 10:28:17 -0500
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> Doug
See comment interspersed...
Using the record above, how would I calculate the following?
>
> 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the
> current number)?
>
Should be less the the current size, if you want the file to merge
> 2) SPLIT - would 90% seem about right?
>
supposed to be bigger than the number of groups (92,776
in this case)?
Chris
> Date: Tue, 3 Jul 2012 11:04:53 -0600
> From: dave...@gmail.com
> To: u2-users@listserver.u2ug.org
> Subject: Re: [U2] RESIZE - dynamic files
>
> Yep, I added an extra 2 in the key value. Oh, the
Yep, I added an extra 2 in the key value. Oh, the perils of cut and
paste...
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
table above I'm just trying to get an idea of what these should be
set at.
Thanks,
Chris
> From: cjausti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Tue, 3 Jul 2012 10:28:17 -0500
> Subject: Re: [U2] RESIZE - dynamic files
>
>
> Doug,
>
>
seem right because if I use that formula I always calculate a
minimum modulus of less than the current modulus.
Thanks,
Chris
> Date: Mon, 2 Jul 2012 16:08:16 -0600
> From: dave...@gmail.com
> To: u2-users@listserver.u2ug.org
> Subject: Re: [U2] RESIZE - dynamic files
>
> H
; Ladybridge Systems Ltd
> 17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
> +44 (0)1604-709200
>
>
>
>
> -Original Message-
> From: u2-users-boun...@listserver.u2ug.org
> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
&g
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Monday, July 02, 2012 2:07 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
The dynamic file I'm working with is below. What do 'overflo
Hi Chris:
You cannot get away with not resizing dynamic files in my experience. The
files do not split and merge like we are led to believe. The separator is
not used on dynamic files. Your Universe file is badly sized. The math
below will get you reasonably file size.
Let's do the math:
258
adly'
#'s go down. Is this the goal when tuning a dynamic file?
Chris
> From: martinphill...@ladybridge.com
> To: u2-users@listserver.u2ug.org
> Date: Mon, 2 Jul 2012 20:56:40 +0100
> Subject: Re: [U2] RESIZE - dynamic files
>
> Hi Chris,
>
> The whole po
sti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Mon, 2 Jul 2012 14:55:21 -0500
> Subject: [U2] RESIZE - dynamic files
>
>
> I was wondering if anyone had instructions on RESIZE with a dynamic file? For
> example I have a file called 'TEST_FILE'
58687736 bytes
Total size of record IDs ... 19283300 bytes
Unused space ... 171626404 bytes
Total space for records 449597440 bytes
Thanks,
Chris
> From: cjausti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Mon, 2 Jul 2012 14:55:21 -0500
> Subject:
Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: 02 July 2012 20:22
To: u2-users@listserver.u2ug.org
Subject: [U2] RESIZE - dynamic files
I was wondering if anyone had instructions on RESIZE with a dynamic
I was wondering if anyone had instructions on RESIZE with a dynamic file? For
example I have a file called 'TEST_FILE'
with the following:
01 ANALYZE.FILE TEST_FILE
File name .. TEST_FILE
Pathname ... TEST_FILE
File type .. DYNAMIC
File style
I was wondering if anyone had instructions on RESIZE with a dynamic file? For
example I have a file called 'TEST_FILE'
with the following:
01 ANALYZE.FILE TEST_FILE
File name .. TEST_FILE
Pathname ... TEST_FILE
File type .. DYNAMIC
File style
[EMAIL PROTECTED] wrote on 06/12/2006 12:57:03 PM:
> What does the guide -r option do ?
>
> We have been using the -a option.
The -r option sends guide output to a hashed file. This makes it very easy
to select for files that are undersized, or that have corruption. So I'll
often do a CREAT
, but I can't imagine that
there is anything better.
Mark Hennessey
(a FAST Customer since 2002)
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Dave S
Sent: Monday, June 12, 2006 10:25 AM
To: u2-users@listserver.u2ug.org
Subject: [U2] RESIZE DYNAMIC FI
What does the guide -r option do ?
We have been using the -a option.
Timothy Snyder <[EMAIL PROTECTED]> wrote:
[EMAIL PROTECTED] wrote on 06/12/2006 10:24:51 AM:
> Does anyone have any tech tips on how to select parameters when
> resizing dynamic files ?
The following is from a published
[EMAIL PROTECTED] wrote on 06/12/2006 10:24:51 AM:
> Does anyone have any tech tips on how to select parameters when
> resizing dynamic files ?
The following is from a published tech tip. It provides guidelines, but
of course the nature of MV files makes it difficult to predict optimal
sizing.
, 2006 10:25 AM
To: u2-users@listserver.u2ug.org
Subject: [U2] RESIZE DYNAMIC FILES
Does anyone have any tech tips on how to select parameters when resizing
dynamic files ?
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protecti
Does anyone have any tech tips on how to select parameters when resizing
dynamic files ?
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---
u2-users mailing list
u2-users@listserver.u2ug
73 matches
Mail list logo