date:20120703

One of our Java programmers lives and breaths Mac's.  We develop on the Mac
and run on the Mac.  We have for years many years been running on the Mac.
 There are many things that have to be tuned for the Mac on Eclipse.
 Whether it is the the Java libraries that are slightly different or SWT,
there is a lot of work that has to be Mac specific.

Regards,
Doug
www.u2logic.com/tools.html
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] UniVerse LIST statement question [not-secure]

2012-07-03 Thread Charles Stevenson


What Brian said, except don't replace LIST!
Rather, write your subroutine to be called from an I-descriptors:

 MARKH
0001   FUNCTION MARKH( INARG )
0002   COMMON /MARKH/ START.IDT, END.IDT, PREV.ID, VLIST
0003 *
0004 * Assumes common has been initialized properly before this function
0005 * is called from i-descriptors
0006 *
0007   IF @ID = PREV.ID ELSE
0008  IDTS = @RECORD 10 
0009  VLIST = 
0010  VMC = 0
0011  VMAX = DCOUNT( IDTS, @VM )
0012  FOR V = 1 TO VMAX
0013 IDT = IDTS1,V
0014 IF IDT = START.IDT THEN IF IDT = END.IDT THEN 
VLIST-1 = V

0015  NEXT V
0016   END
0017 *
0018   OUTARG = 
0019   MORE = LEN( VLIST ); * true/false
0020   LOOP WHILE MORE
0021  REMOVE V FROM VLIST SETTING MORE
0022  OUTARG1,-1 = INARG1, V 
0023   REPEAT
0024   SETREM 0 ON VLIST
0025 *
0026   RETURN( OUTARG )
0027END

An example using it is below.

It's more flexible and maintainable in the long run than tossing 
RETRIEVE altogether.
(Actually, I do think it can be done with the mv-handling functions but 
an I-descriptor subroutine will be more maintainable.)


Your sub should limits the output to the values you want. If every value 
is out of range, your subr returns null.  It can be called for various 
associated fields, and the 1st time called for a given @ID it figures 
out the value range, and saves to labeled common what vals you want for 
that record.  Subsequent calls to the sub, as long as @ID hasn't 
changed, gets the already calculated value range from common, and 
applies it to the field you pass it.


The only trick is feeding your subroutine the date range you won't know 
until runtime.I don't know a good way to feed an I-descriptor 
command line parameters such as = 6/1/2012 = 6/30/2012
You'll need to set them in a control record or, better, in @USER1, 
@USER2, or in your labeled common before the LIST is executed.


Use the I-descriptors that call your subroutine as output criteria and 
not WITH or WHEN selection criteria.  The guts of the subr will do the 
selection limits.


That may not be altogether clear.  Here's an example.
1st the relevant dict entries then an example:


DICT RTAG09:45:22am  03 Jul 2012  Page1

Field. Type  Field Conversion.. Column. Output 
Depth 
Name.. Field. Definition... Code Heading Format 
Assoc..

   Number

LOCD3 12LS
EVENT.CODE D8Event 4L M EVENT
.ASSOC
 Code
EVENT.DT   D   10   D2/  Event 8R M EVENT
.ASSOC
 Date
PRINTERD   13 15LS
MARKH.CODE I  SUBR(  Event 4L M MARKH
'MARKH',  .ASSOC
  EVENT.CODE )
 Code
MARKH.DT   I  SUBR( D2/  Event 8R M MARKH
'MARKH',  .ASSOC
  EVENT.DT )
 Date
MARKH.ASSOCPH MARKH.LINO
  MARKH.CODE
  MARKH.REF
  MARKH.DT
  MARKH.TM
  MARKH.WHO
  MARKH.DTM

7 records listed.


RUN CDS.BP MARKH.INIT 4/1 4/30
LIST RTAG LOC PRINTER   EVENT.CODE EVENT.DT   MARKH.CODE MARKH.DT


LIST RTAG LOC PRINTER EVENT.CODE EVENT.DT MARKH.CODE MARKH.DT 
09:46:05am  03 Jul 2012  PAGE1

Return.. LOC. PRINTER Event Event... Event Event...
Tag.  Code. Date Code. Date

 5137176 TPAWHSE-IN   ZEBRATPAPRINT 02/08/12
  NEW   11/04/11
10009741 EROCWHSE-OUT PI04/26/12 PI 04/26/12
  NEW   04/26/12 NEW 04/26/12
 5135103 PROCWHSE-DEF REQ   12/30/11
  REQ   12/30/11
  ASGN  12/30/11
  FREE  11/03/11
 5134267 EROCDPO-DIN  ZEBRATPAI 03/01/12
  PRINT 02/08/12
  REQ   12/09/11
  ASGN  12/09/11
  FREE  11/03/11
10010396 DFWWHSE  PI05/22/12
  NEW   05/22/12
 5119929  SI04/10/12 SI 04/10/12
  NEW   09/20/11
10004562 CROCWHSE-OUT ZEBRAEROC   PRINT 04/04/12 PRINT 04/04/12
  PRINT 01/31/12
  PRINT 01/24/12
  PRINT

[U2] Exiting UniData

2012-07-03 Thread Israel, John R.

Going back to my post about runaway processes:
According to our UniData manuals, the official commands to terminate a UniData 
session are BYE, LO and QUIT.

I have since found that LO and OFF (not sure where this came from) simply run 
the QUIT command.

QUIT runs an SB command called SH.OFF (which we obviously do not have the 
source code to review).

BYE still runs the verb BYE, so nothing different there.


We are using Avanté which obviously is built with SB.

My question is this: do ALL SB sites change the value of QUIT to run SH.OFF, or 
was this something that Epicor does?

If anyone is using SB but NOT using an Epicor product, what does your QUIT 
command do?


Thanks

John
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] Exiting UniData

2012-07-03 Thread Dave Davis

Runs the globally cataloged program _SB_SH.OFF

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Israel, John R.
Sent: Tuesday, July 03, 2012 10:31 AM
To: 'U2 Users List'
Subject: [U2] Exiting UniData

Going back to my post about runaway processes:
According to our UniData manuals, the official commands to terminate a UniData 
session are BYE, LO and QUIT.

I have since found that LO and OFF (not sure where this came from) simply run 
the QUIT command.

QUIT runs an SB command called SH.OFF (which we obviously do not have the 
source code to review).

BYE still runs the verb BYE, so nothing different there.

We are using Avanté which obviously is built with SB.

My question is this: do ALL SB sites change the value of QUIT to run SH.OFF, or 
was this something that Epicor does?

If anyone is using SB but NOT using an Epicor product, what does your QUIT 
command do?

Thanks

John
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Dave Davis
Team Lead, RD

P: 614-875-4910 x108
F: 614-875-4088
E: dda...@harriscomputer.com
[http://www.harriscomputer.com/images/signatures/HarrisSchools.gif]

[http://www.harriscomputer.com/images/signatures/DivisionofHarris.gif]http://www.harriscomputer.com/
6110 Enterprise Parkway
Grove City, OH
43123
www.harris-schoolsolutions.comhttp://www.harris-schoolsolutions.com

This message is intended exclusively for the individual or entity to which it 
is addressed. This communication may contain information that is proprietary, 
privileged or confidential or otherwise legally exempt from disclosure. If you 
are not the named addressee, you are not authorized to read, print, retain, 
copy or disseminate this message or any part of it. If you have received this 
message in error, please notify the sender immediately by e-mail and delete all 
copies of the message.

___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] UniVerse LIST statement question [not-secure]

2012-07-03 Thread Charles Stevenson

Ed is quite right.

1: I just did it for simplicity in the example.

2: @SENTENCE, without a wrapper: you have do get ReTrieve to ignore
part of the sentence. I've never come up with a good way to do that
other than via EVAL. Anyone?

3. EVAL: Yes, it is a way to pass run-time parameters!
I tend to forget about it because I - personally - don't like it in
production.

EVAL means dictionaries can't be read-only.
And they tend to permanently leave temporary I-descriptors scattered
in the dicts if when something terminates abnormally.

RetrieVe rudely spits the EVAL compilation report into the output, too.
If you have multiple associated columns, if your usage gets fancy
(that (future) option is a reason for this approach), then your EVAL may
need ASSOC, ASSOC.WITH, MULTIVALUED keywords, besides AS, FMT, COL.HDG.

Other than that, I love EVAL.

On 7/3/2012 11:08 AM, Ed Clark wrote:

There are 3 ways I can think of offhand to tell the subroutine what date range
to use. I like #3 the best:

1: use common variables, which is what the example function MARKH does. The
downside is that you have to assign the common variables before you run the
query.

2: You could use @SENTENCE in the function and parse out the date range. That
could be easy or hard depending on how many people write queries and what odd
contortions of syntax they use. But both #1 and #2 could be simplified by
writing a wrapper program. An example command line would look like:
QLAUNCH 06/01/2012 06/30/2012 ~ LIST MYFILE INRANGE
QLAUNCH parses arguments before the '~` delimiter and puts them into a common.
The INRANGE attribute calls the function MARKH which uses the common.

3: use the function directly from the command line with EVAL:
LIST MYFILE EVAL SUBR('MARKH','06/01/2012','06/30/2012') AS MYCOL COL.HDG
WHATEVER FMT 11L
It's wordier, but you have very fine-grained control over what comes out on the
report. COL.HDG, and FMT are optional and AS is optional in this case. You
would us it if you wanted to sort or break on the EVAL column.

On Jul 3, 2012, at 9:55 AM, Charles Stevenson wrote:

What Brian said, except don't replace LIST!
Rather, write your subroutine to be called from an I-descriptors:

MARKH
0001 FUNCTION MARKH( INARG )
0002 COMMON /MARKH/ START.IDT, END.IDT, PREV.ID, VLIST
0003 *
0004 * Assumes common has been initialized properly before this function
0005 * is called from i-descriptors
0006 *
0007 IF @ID = PREV.ID ELSE
0008 IDTS = @RECORD 10
0009 VLIST =
0010 VMC = 0
0011 VMAX = DCOUNT( IDTS, @VM )
0012 FOR V = 1 TO VMAX
0013 IDT = IDTS1,V
0014 IF IDT = START.IDT THEN IF IDT = END.IDT THEN VLIST-1 = V
0015 NEXT V
0016 END
0017 *
0018 OUTARG =
0019 MORE = LEN( VLIST ); * true/false
0020 LOOP WHILE MORE
0021 REMOVE V FROM VLIST SETTING MORE
0022 OUTARG1,-1 = INARG1, V
0023 REPEAT
0024 SETREM 0 ON VLIST
0025 *
0026 RETURN( OUTARG )
0027END

An example using it is below.

It's more flexible and maintainable in the long run than tossing RETRIEVE
altogether.
(Actually, I do think it can be done with the mv-handling functions but an
I-descriptor subroutine will be more maintainable.)

Your sub should limits the output to the values you want. If every value is out
of range, your subr returns null. It can be called for various associated
fields, and the 1st time called for a given @ID it figures out the value range,
and saves to labeled common what vals you want for that record. Subsequent
calls to the sub, as long as @ID hasn't changed, gets the already calculated
value range from common, and applies it to the field you pass it.

The only trick is feeding your subroutine the date range you won't know until runtime.I don't
know a good way to feed an I-descriptor command line parameters such as = 6/1/2012
= 6/30/2012
You'll need to set them in a control record or, better, in @USER1, @USER2, or
in your labeled common before the LIST is executed.

Use the I-descriptors that call your subroutine as output criteria and not WITH
or WHEN selection criteria. The guts of the subr will do the selection limits.

That may not be altogether clear. Here's an example.
1st the relevant dict entries then an example:

DICT RTAG09:45:22am 03 Jul 2012 Page1

Field. Type Field Conversion.. Column. Output Depth
Name.. Field. Definition... Code Heading Format Assoc..
Number

LOCD3 12LS
EVENT.CODE D8Event 4L M EVENT
.ASSOC
Code
EVENT.DT D 10 D2/ Event 8R M EVENT
.ASSOC
Date
PRINTERD 13 15LS
MARKH.CODE I SUBR( Event 4L

Re: [U2] RESIZE - dynamic files


File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
28229 overflowed, 2485 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   500600832 bytes
Total size of record data ..   287035391 bytes
Total size of record IDs ...   21508449 bytes
Unused space ...   192048800 bytes
Total space for records    500592640 bytes

Using the record above, how would I calculate the following?

1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
current number)?
2) SPLIT - would 90% seem about right?
3) MERGE - would 20% seem about right? 
4) Large Record Size - does 3276 seem right? 
5) Group Size - should I be using 4096?

I'm just a bit confused as to how to set these, I saw the formula to calculate 
the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I always get a 
lower number
than my current modulus.. 

I also saw where it said to simply take your current modulus # and add 10-20% 
and set the MINIMUM.MODULUS based on that..

Based on the table above I'm just trying to get an idea of what these should be 
set at.

Thanks,

Chris


 From: cjausti...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:28:17 -0500
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Doug,
 
 When I do the math I come up with a different # (see below):
 
 File name ..   TEST_FILE
 Pathname ...   TEST_FILE
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    82850 current ( minimum 24, 104 empty,
 26225 overflowed, 1441 badly )
 Number of records ..   1157122
 Large record size ..   2036 bytes
 Number of large records    576
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   449605632 bytes
 Total size of record data ..   258687736 bytes
 Total size of record IDs ...   19283300 bytes
 Unused space ...   171626404 bytes
 Total space for records    449597440 bytes
 
 -- 
 258,687,736 bytes - Total size of record data
 19,283,300 bytes - Total size of record IDs
 ===
 277,971,036 bytes (record + id's)
 
 277,971,036 / 4,084 = 68,063 bytes (minimum modulus)
 -- 
 
 68,063 is less than the current modulus of 82,850. Something with this 
 formula doesn't seem right because if I use that formula I always calculate a 
 minimum modulus of less than the current modulus.
 
 Thanks,
 
 Chris
 
 
 
  Date: Mon, 2 Jul 2012 16:08:16 -0600
  From: dave...@gmail.com
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  Hi Chris:
  
  You cannot get away with not resizing dynamic files in my experience.  The
  files do not split and merge like we are led to believe.  The separator is
  not used on dynamic files.  Your Universe file is badly sized.  The math
  below will get you reasonably file size.
  
  Let's do the math:
  
  258687736 (Record Size)
  192283300 (Key Size)
  
  450,971,036 (Data and Key Size)
  
  4096 (Group Size)
  - 12   (32 Bit Overhead)
  
  4084 Usable Space
  
  450971036/4084 = Minimum Modulo 110424 (Prime is 110431)
  
  
  [ad]
  I hate doing this math all of the time.  I have a reasonably priced resize
  program called XLr8Resizer for $99.00 to do this for me.
  [/ad]
  
  Regards,
  Doug
  www.u2logic.com/tools.html
  XLr8Resizer for the rest of us
  ___
  U2-Users mailing list
  U2-Users@listserver.u2ug.org
  http://listserver.u2ug.org/mailman/listinfo/u2-users
 
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

Yep, I added an extra 2 in the key value.  Oh, the perils of cut and
paste...
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files


No worries Doug. I'm just wondering if the calculation makes sense (if we use 
the example below):

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
28229 overflowed, 2485 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   500600832 bytes
Total size of record data ..   287035391 bytes
Total size of record IDs ...   21508449 bytes
Unused space ...   192048800 bytes
Total space for records    500592640 bytes

FORMULA - (287,035,391+21,508,449) / (4,084) = 75,549  MINIMUM.MODULUS

The question I have is whether 75,549 makes sense for this record. I thought 
the MINIMUM.MODULUS was supposed to be bigger than the number of groups (92,776 
in this case)?

Chris


 Date: Tue, 3 Jul 2012 11:04:53 -0600
 From: dave...@gmail.com
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 Yep, I added an extra 2 in the key value.  Oh, the perils of cut and
 paste...
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

See comment interspersed...

Using the record above, how would I calculate the following?

 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the
 current number)?

Should be less the the current size, if you want the file to merge


 2) SPLIT - would 90% seem about right?

Depends on the history of the file.  Is the data growing over time?  The
way the file looks now the split should be reduced because you have 31% in
overflow.

3) MERGE - would 20% seem about right?

Won't be used on a growth file, so the file history is important.

4) Large Record Size - does 3276 seem right?

Can be calculated with a lot of effort, but yield little gain.

5) Group Size - should I be using 4096?

You have two group sizes on dynamic files 2048 and 4096.  If you lower it
you need to double your modulo, roughly.  If you keep it the same you need
to increase your modulo because 31% of your file is in overflow.



___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

(record + id / 4096 or 2048)

You need to factor in overhead  the split factor:   (records + ids) * 1.1 * 
1.25  / 4096(for 80%) 

If you use a 20% merge factor and a 80% split factor, the file will start 
merging unless you delete 60 percent of your records.  If you use 90% split 
factor, you will have more overflowed groups.  These numbers refer to the total 
amount of data in the file, not to any individual group.

For records of the size that you have, I do not see any advantage to using a 
larger, 4096, group size. You will end up with twice the number of records per 
group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.

-Rick

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 9:48 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
28229 overflowed, 2485 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   500600832 bytes
Total size of record data ..   287035391 bytes
Total size of record IDs ...   21508449 bytes
Unused space ...   192048800 bytes
Total space for records    500592640 bytes

Using the record above, how would I calculate the following?

1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
current number)?
2) SPLIT - would 90% seem about right?
3) MERGE - would 20% seem about right? 
4) Large Record Size - does 3276 seem right? 
5) Group Size - should I be using 4096?

I'm just a bit confused as to how to set these, I saw the formula to calculate 
the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I always get a 
lower number
than my current modulus.. 

I also saw where it said to simply take your current modulus # and add 10-20% 
and set the MINIMUM.MODULUS based on that..

Based on the table above I'm just trying to get an idea of what these should be 
set at.

Thanks,

Chris


 From: cjausti...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:28:17 -0500
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Doug,
 
 When I do the math I come up with a different # (see below):
 
 File name ..   TEST_FILE
 Pathname ...   TEST_FILE
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    82850 current ( minimum 24, 104 empty,
 26225 overflowed, 1441 badly )
 Number of records ..   1157122
 Large record size ..   2036 bytes
 Number of large records    576
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   449605632 bytes
 Total size of record data ..   258687736 bytes
 Total size of record IDs ...   19283300 bytes
 Unused space ...   171626404 bytes
 Total space for records    449597440 bytes
 
 -- 
 258,687,736 bytes - Total size of record data
 19,283,300 bytes - Total size of record IDs
 ===
 277,971,036 bytes (record + id's)
 
 277,971,036 / 4,084 = 68,063 bytes (minimum modulus)
 -- 
 
 68,063 is less than the current modulus of 82,850. Something with this 
 formula doesn't seem right because if I use that formula I always calculate a 
 minimum modulus of less than the current modulus.
 
 Thanks,
 
 Chris
 
 
 
  Date: Mon, 2 Jul 2012 16:08:16 -0600
  From: dave...@gmail.com
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  Hi Chris:
  
  You cannot get away with not resizing dynamic files in my experience.  The
  files do not split and merge like we are led to believe.  The separator is
  not used on dynamic files.  Your Universe file is badly sized.  The math
  below will get you reasonably file size.
  
  Let's do the math:
  
  258687736 (Record Size)
  192283300 (Key Size)
  
  450,971,036 (Data and Key Size)
  
  4096 (Group Size)
  - 12   (32 Bit Overhead)
  
  4084 Usable Space
  
  450971036/4084 = Minimum Modulo 110424 (Prime is 110431)
  
  
  [ad]
  I hate doing this math all of the time.  I have a reasonably priced resize
  program called XLr8Resizer for $99.00 to do this for me.
  [/ad]
  
  Regards,
  Doug
  www.u2logic.com/tools.html
  XLr8Resizer for the rest of us
  ___
  U2-Users

Re: [U2] Running XLr8 Tools inside U2 DBTools new Eclipse release

Hi Dan and others:

I finished testing all of the Rocket tools.  Turns out the only one that
really works as a plug-in platform is BDT for U2logic plug-ins.  The other
products are crippled versions of Eclipse with limited menus by design.

Another big problem is that Rocket's Eclipse products do not share work
spaces, so if you start BDT your work space is in that directory.  If you
start U2Admin the work space is in that directory.  Unfortunately this
situation is very messy for U2logic, a Rocket VAR, to support.  I'm going
to have fun explaining this to the first client that asks why the programs
don't share the server names.

Regards,
Doug
www.u2logic.com
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files


So for this example what would be a good SPLIT level and what would be a good 
MERGE level to use? It was my understanding that I wanted to lower my merge to 
something below 50% and
increase the split to reduce splitting.

Chris


 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:21:16 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 (record + id / 4096 or 2048)
 
 You need to factor in overhead  the split factor:   (records + ids) * 1.1 * 
 1.25  / 4096(for 80%) 
 
 If you use a 20% merge factor and a 80% split factor, the file will start 
 merging unless you delete 60 percent of your records.  If you use 90% split 
 factor, you will have more overflowed groups.  These numbers refer to the 
 total amount of data in the file, not to any individual group.
 
 For records of the size that you have, I do not see any advantage to using a 
 larger, 4096, group size. You will end up with twice the number of records 
 per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
 
 -Rick
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 9:48 AM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
 28229 overflowed, 2485 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   500600832 bytes
 Total size of record data ..   287035391 bytes
 Total size of record IDs ...   21508449 bytes
 Unused space ...   192048800 bytes
 Total space for records    500592640 bytes
 
 Using the record above, how would I calculate the following?
 
 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
 current number)?
 2) SPLIT - would 90% seem about right?
 3) MERGE - would 20% seem about right? 
 4) Large Record Size - does 3276 seem right? 
 5) Group Size - should I be using 4096?
 
 I'm just a bit confused as to how to set these, I saw the formula to 
 calculate the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I 
 always get a lower number
 than my current modulus.. 
 
 I also saw where it said to simply take your current modulus # and add 10-20% 
 and set the MINIMUM.MODULUS based on that..
 
 Based on the table above I'm just trying to get an idea of what these should 
 be set at.
 
 Thanks,
 
 Chris
 
 
  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 10:28:17 -0500
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  Doug,
  
  When I do the math I come up with a different # (see below):
  
  File name ..   TEST_FILE
  Pathname ...   TEST_FILE
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    82850 current ( minimum 24, 104 empty,
  26225 overflowed, 1441 badly )
  Number of records ..   1157122
  Large record size ..   2036 bytes
  Number of large records    576
  Group size .   4096 bytes
  Load factors ...   80% (split), 50% (merge) and 80% (actual)
  Total size .   449605632 bytes
  Total size of record data ..   258687736 bytes
  Total size of record IDs ...   19283300 bytes
  Unused space ...   171626404 bytes
  Total space for records    449597440 bytes
  
  -- 
  258,687,736 bytes - Total size of record data
  19,283,300 bytes - Total size of record IDs
  ===
  277,971,036 bytes (record + id's)
  
  277,971,036 / 4,084 = 68,063 bytes (minimum modulus)
  -- 
  
  68,063 is less than the current modulus of 82,850. Something with this 
  formula doesn't seem right because if I use that formula I always calculate 
  a 
  minimum modulus of less than the current modulus.
  
  Thanks,
  
  Chris
  
  
  
   Date: Mon, 2 Jul 2012 16:08:16 -0600
   From: dave...@gmail.com
   To: u2-users@listserver.u2ug.org
   Subject: Re: [U2] RESIZE - dynamic files
   
   Hi Chris:
   
   You cannot get away with not resizing dynamic files in my experience.  The
   files do not split and merge like we are led to believe.  The separator is
   not used on dynamic files.  Your Universe file is badly sized.  The math
   below will get you reasonably file size.
   
   Let's do the math:
   
   258687736 (Record Size)

Re: [U2] RESIZE - dynamic files



Doug,

The data is growing over time with this file. Does that mean I should ignore 
the formula? Or should I still use a lower MINIMUM.MODULO than the
actual modulo #..

Is the idea to reduce overflow by lowering the split? What is this 'overflow' 
referring to?

  2) SPLIT - would 90% seem about right?
 
 Depends on the history of the file.  Is the data growing over time?  The
 way the file looks now the split should be reduced because you have 31% in
 overflow.

So basically don't spend much time worrying about large record size?

 4) Large Record Size - does 3276 seem right?
 
 Can be calculated with a lot of effort, but yield little gain.

This seems like a moot point as well, as long as the ratio in regards to the 
MINIMUM.MODULO is set proportionally?

 5) Group Size - should I be using 4096?
 
 You have two group sizes on dynamic files 2048 and 4096.  If you lower it
 you need to double your modulo, roughly.  If you keep it the same you need
 to increase your modulo because 31% of your file is in overflow.

Chris


  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files


Using the formula below, and changing the split to 90% I get the following:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
22249 overflowed, 1764 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 72% (actual)
Total size .   519921664 bytes
Total size of record data ..   287400591 bytes
Total size of record IDs ...   21508497 bytes
Unused space ...   211004384 bytes
Total space for records    519913472 bytes

How does this look in terms of performance? 

My Actual load went down 8% as well as some overflow but it looks like my load 
% is still high at 72% I'm wondering if I should raise the MINIMUM.MODULUS even 
more 
since I still have a decent amount of overflow and not many large records. 

Chris


 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:21:16 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 (record + id / 4096 or 2048)
 
 You need to factor in overhead  the split factor:   (records + ids) * 1.1 * 
 1.25  / 4096(for 80%) 
 
 If you use a 20% merge factor and a 80% split factor, the file will start 
 merging unless you delete 60 percent of your records.  If you use 90% split 
 factor, you will have more overflowed groups.  These numbers refer to the 
 total amount of data in the file, not to any individual group.
 
 For records of the size that you have, I do not see any advantage to using a 
 larger, 4096, group size. You will end up with twice the number of records 
 per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
 
 -Rick
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 9:48 AM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
 28229 overflowed, 2485 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   500600832 bytes
 Total size of record data ..   287035391 bytes
 Total size of record IDs ...   21508449 bytes
 Unused space ...   192048800 bytes
 Total space for records    500592640 bytes
 
 Using the record above, how would I calculate the following?
 
 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
 current number)?
 2) SPLIT - would 90% seem about right?
 3) MERGE - would 20% seem about right? 
 4) Large Record Size - does 3276 seem right? 
 5) Group Size - should I be using 4096?
 
 I'm just a bit confused as to how to set these, I saw the formula to 
 calculate the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I 
 always get a lower number
 than my current modulus.. 
 
 I also saw where it said to simply take your current modulus # and add 10-20% 
 and set the MINIMUM.MODULUS based on that..
 
 Based on the table above I'm just trying to get an idea of what these should 
 be set at.
 
 Thanks,
 
 Chris
 
 
  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 10:28:17 -0500
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  Doug,
  
  When I do the math I come up with a different # (see below):
  
  File name ..   TEST_FILE
  Pathname ...   TEST_FILE
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    82850 current ( minimum 24, 104 empty,
  26225 overflowed, 1441 badly )
  Number of records ..   1157122
  Large record size ..   2036 bytes
  Number of large records    576
  Group size .   4096 bytes
  Load factors ...   80% (split), 50% (merge) and 80% (actual)
  Total size .   449605632 bytes
  Total size of record data ..   258687736 bytes
  Total size of record IDs ...   19283300 bytes
  Unused space ...   171626404 bytes
  Total space for records    449597440

Re: [U2] RESIZE - dynamic files

2012-07-03 Thread Jeff Schasny

I would recommend that if you intend to do resizing on a regular basis 
an you want to improve the performance of the file you might consider 
resizing the file to a static file type so that you can have more 
control over the hashing algorithm, separation and modulo.


Chris Austin wrote:

Using the formula below, and changing the split to 90% I get the following:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
22249 overflowed, 1764 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 72% (actual)
Total size .   519921664 bytes
Total size of record data ..   287400591 bytes
Total size of record IDs ...   21508497 bytes
Unused space ...   211004384 bytes
Total space for records    519913472 bytes

How does this look in terms of performance? 

My Actual load went down 8% as well as some overflow but it looks like my load % is still high at 72% I'm wondering if I should raise the MINIMUM.MODULUS even more 
since I still have a decent amount of overflow and not many large records. 


Chris


  

From: r...@lynden.com
To: u2-users@listserver.u2ug.org
Date: Tue, 3 Jul 2012 10:21:16 -0700
Subject: Re: [U2] RESIZE - dynamic files

(record + id / 4096 or 2048)

You need to factor in overhead  the split factor:   (records + ids) * 1.1 * 1.25  / 4096(for 80%) 


If you use a 20% merge factor and a 80% split factor, the file will start 
merging unless you delete 60 percent of your records.  If you use 90% split 
factor, you will have more overflowed groups.  These numbers refer to the total 
amount of data in the file, not to any individual group.

For records of the size that you have, I do not see any advantage to using a 
larger, 4096, group size. You will end up with twice the number of records per 
group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.

-Rick

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 9:48 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
28229 overflowed, 2485 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   500600832 bytes
Total size of record data ..   287035391 bytes
Total size of record IDs ...   21508449 bytes
Unused space ...   192048800 bytes
Total space for records    500592640 bytes

Using the record above, how would I calculate the following?

1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
current number)?
2) SPLIT - would 90% seem about right?
3) MERGE - would 20% seem about right? 
4) Large Record Size - does 3276 seem right? 
5) Group Size - should I be using 4096?


I'm just a bit confused as to how to set these, I saw the formula to calculate 
the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I always get a 
lower number
than my current modulus.. 


I also saw where it said to simply take your current modulus # and add 10-20% 
and set the MINIMUM.MODULUS based on that..

Based on the table above I'm just trying to get an idea of what these should be 
set at.

Thanks,

Chris




From: cjausti...@hotmail.com
To: u2-users@listserver.u2ug.org
Date: Tue, 3 Jul 2012 10:28:17 -0500
Subject: Re: [U2] RESIZE - dynamic files


Doug,

When I do the math I come up with a different # (see below):

File name ..   TEST_FILE
Pathname ...   TEST_FILE
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    82850 current ( minimum 24, 104 empty,
26225 overflowed, 1441 badly )
Number of records ..   1157122
Large record size ..   2036 bytes
Number of large records    576
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   449605632 bytes
Total

Re: [U2] RESIZE - dynamic files

The split load is not affecting anything here, since it is more than the actual 
load.  What your overflow suggests is that you lower the split.load value to 
70$% or below.  You could go ahead and set the merge.load to an arbitrarily low 
number (1), and it will probably never do a merge, which would be the same as 
specifying a minimum.modulus equal to as large as it ever gets.  The 
exception to this is during file creation  clear.file,  when the 
minimum.modulus value determines the initial disk allocation.  Short of going 
to an arbitrarily large minimum.modulus, and a very low split.load, you are 
going to have some overflow (unless you have sequential keys  like sized 
records).

-Rick

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 12:54 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


Using the formula below, and changing the split to 90% I get the following:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
22249 overflowed, 1764 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 72% (actual)
Total size .   519921664 bytes
Total size of record data ..   287400591 bytes
Total size of record IDs ...   21508497 bytes
Unused space ...   211004384 bytes
Total space for records    519913472 bytes

How does this look in terms of performance? 

My Actual load went down 8% as well as some overflow but it looks like my load 
% is still high at 72% I'm wondering if I should raise the MINIMUM.MODULUS even 
more 
since I still have a decent amount of overflow and not many large records. 

Chris


 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:21:16 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 (record + id / 4096 or 2048)
 
 You need to factor in overhead  the split factor:   (records + ids) * 1.1 * 
 1.25  / 4096(for 80%) 
 
 If you use a 20% merge factor and a 80% split factor, the file will start 
 merging unless you delete 60 percent of your records.  If you use 90% split 
 factor, you will have more overflowed groups.  These numbers refer to the 
 total amount of data in the file, not to any individual group.
 
 For records of the size that you have, I do not see any advantage to using a 
 larger, 4096, group size. You will end up with twice the number of records 
 per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
 
 -Rick
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 9:48 AM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    92776 current ( minimum 31, 89 empty,
 28229 overflowed, 2485 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   500600832 bytes
 Total size of record data ..   287035391 bytes
 Total size of record IDs ...   21508449 bytes
 Unused space ...   192048800 bytes
 Total space for records    500592640 bytes
 
 Using the record above, how would I calculate the following?
 
 1) MINIMUM.MODULUS (Is there a formula to use or should I add 20% to the 
 current number)?
 2) SPLIT - would 90% seem about right?
 3) MERGE - would 20% seem about right? 
 4) Large Record Size - does 3276 seem right? 
 5) Group Size - should I be using 4096?
 
 I'm just a bit confused as to how to set these, I saw the formula to 
 calculate the MINIMUM.MODULUS which is (record + id / 4096 or 2048) but I 
 always get a lower number
 than my current modulus.. 
 
 I also saw where it said to simply take your current modulus # and add 10-20% 
 and set the MINIMUM.MODULUS based on that..
 
 Based on the table above I'm just trying to get an idea of what these should 
 be set at.
 
 Thanks,
 
 Chris
 
 
  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 10:28:17 -0500
  Subject: Re:

Re: [U2] UniVerse LIST statement question [not-secure]

2012-07-03 Thread Ed Clark

There have been more than a few times when I wished that MV query was a little 
more extensible. On a couple of occasions I have created throwaway connectives 
as a means to pass flags to an i-type or a B correlative, but it would be nice 
if there were a keyword that flagged to just ignore everything after it on the 
line.

Hmm, I never knew that EVAL needed to write to the dict on universe. Is that 
try on unidata as well?

On Jul 3, 2012, at 12:47 PM, Charles Stevenson wrote:

 Ed is quite right.
 
 1:  I just did it for simplicity in the example.
 
 2:  @SENTENCE, without a wrapper:  you have do get ReTrieve to ignore part of 
 the sentence.   I've never come up with a good way to do that other than via 
 EVAL.  Anyone?
 
 3.  EVAL: Yes, it is a way to pass run-time parameters!
 I tend to forget about it because I - personally - don't like it in 
 production.
 EVAL means dictionaries can't be read-only.
 And they tend to permanently leave temporary I-descriptors scattered in the 
 dicts if  when something terminates abnormally.
 RetrieVe rudely spits the EVAL compilation report into the output, too.
 If you have multiple associated columns,  if your usage gets fancy (that 
 (future) option is a reason for this approach), then your EVAL may need 
 ASSOC, ASSOC.WITH, MULTIVALUED keywords, besides AS, FMT, COL.HDG.
 Other than that, I love EVAL.
 
 
 On 7/3/2012 11:08 AM, Ed Clark wrote:
 There are 3 ways I can think of offhand to tell the subroutine what date 
 range to use. I like #3 the best:
 
 1: use common variables, which is what the example function MARKH does. The 
 downside is that you have to assign the common variables before you run the 
 query.
 
 2: You could use @SENTENCE in the function and parse out the date range. 
 That could be easy or hard depending on how many people write queries and 
 what odd contortions of syntax they use. But both #1 and #2 could be 
 simplified by writing a wrapper program. An example command line would look 
 like:
   QLAUNCH 06/01/2012 06/30/2012 ~ LIST MYFILE INRANGE
 QLAUNCH parses arguments before the '~` delimiter and puts them into a 
 common. The INRANGE attribute calls the function MARKH which uses the common.
 
 3: use the function directly from the command line with EVAL:
   LIST MYFILE EVAL SUBR('MARKH','06/01/2012','06/30/2012')  AS MYCOL 
 COL.HDG WHATEVER FMT 11L
 It's wordier, but you have very fine-grained control over what comes out on 
 the report. COL.HDG, and FMT are optional and AS is optional in this case. 
 You would us it if you wanted to sort or break on the EVAL column.
 
 
 
 On Jul 3, 2012, at 9:55 AM, Charles Stevenson wrote:
 
 What Brian said, except don't replace LIST!
 Rather, write your subroutine to be called from an I-descriptors:
 
 MARKH
 0001   FUNCTION MARKH( INARG )
 0002   COMMON /MARKH/ START.IDT, END.IDT, PREV.ID, VLIST
 0003 *
 0004 * Assumes common has been initialized properly before this function
 0005 * is called from i-descriptors
 0006 *
 0007   IF @ID = PREV.ID ELSE
 0008  IDTS = @RECORD 10 
 0009  VLIST = 
 0010  VMC = 0
 0011  VMAX = DCOUNT( IDTS, @VM )
 0012  FOR V = 1 TO VMAX
 0013 IDT = IDTS1,V
 0014 IF IDT = START.IDT THEN IF IDT = END.IDT THEN VLIST-1 
 = V
 0015  NEXT V
 0016   END
 0017 *
 0018   OUTARG = 
 0019   MORE = LEN( VLIST ); * true/false
 0020   LOOP WHILE MORE
 0021  REMOVE V FROM VLIST SETTING MORE
 0022  OUTARG1,-1 = INARG1, V 
 0023   REPEAT
 0024   SETREM 0 ON VLIST
 0025 *
 0026   RETURN( OUTARG )
 0027END
 
 An example using it is below.
 
 It's more flexible and maintainable in the long run than tossing RETRIEVE 
 altogether.
 (Actually, I do think it can be done with the mv-handling functions but an 
 I-descriptor subroutine will be more maintainable.)
 
 Your sub should limits the output to the values you want. If every value is 
 out of range, your subr returns null.  It can be called for various 
 associated fields, and the 1st time called for a given @ID it figures out 
 the value range, and saves to labeled common what vals you want for that 
 record.  Subsequent calls to the sub, as long as @ID hasn't changed, gets 
 the already calculated value range from common, and applies it to the field 
 you pass it.
 
 The only trick is feeding your subroutine the date range you won't know 
 until runtime.I don't know a good way to feed an I-descriptor command 
 line parameters such as = 6/1/2012 = 6/30/2012
 You'll need to set them in a control record or, better, in @USER1, @USER2, 
 or in your labeled common before the LIST is executed.
 
 Use the I-descriptors that call your subroutine as output criteria and not 
 WITH or WHEN selection criteria.  The guts of the subr will do the 
 selection limits.
 
 That may not be altogether clear.  Here's an example.
 1st the relevant dict entries then an example:
 
 
 DICT RTAG

Re: [U2] RESIZE - dynamic files


I guess what I need to know is what's an acceptable % of overflow for a dynamic 
file? For example, when I change the SPLIT LOAD to 90% (while using the 
calculated min modulus)
I'm still left with ~ 20% of overflow (see below). Is 20% overflow considered 
acceptable on average or should I keep tinkering with it to reach a lower 
overflow %?

Correct me if I'm wrong but it seems the goal here is to REDUCE the overflow % 
while not creating too many modulus (groups). 

Chris


File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
21092 overflowed, 1452 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 70% (actual)
Total size .   522260480 bytes
Total size of record data ..   287400239 bytes
Total size of record IDs ...   21508521 bytes
Unused space ...   213343528 bytes
Total space for records    522252288 bytes

 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 13:10:43 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 The split load is not affecting anything here, since it is more than the 
 actual load.  What your overflow suggests is that you lower the split.load 
 value to 70$% or below.  You could go ahead and set the merge.load to an 
 arbitrarily low number (1), and it will probably never do a merge, which 
 would be the same as specifying a minimum.modulus equal to as large as it 
 ever gets.  The exception to this is during file creation  clear.file,  
 when the minimum.modulus value determines the initial disk allocation.  Short 
 of going to an arbitrarily large minimum.modulus, and a very low split.load, 
 you are going to have some overflow (unless you have sequential keys  like 
 sized records).
 
 -Rick
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 12:54 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Using the formula below, and changing the split to 90% I get the following:
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
 22249 overflowed, 1764 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 72% (actual)
 Total size .   519921664 bytes
 Total size of record data ..   287400591 bytes
 Total size of record IDs ...   21508497 bytes
 Unused space ...   211004384 bytes
 Total space for records    519913472 bytes
 
 How does this look in terms of performance? 
 
 My Actual load went down 8% as well as some overflow but it looks like my 
 load % is still high at 72% I'm wondering if I should raise the 
 MINIMUM.MODULUS even more 
 since I still have a decent amount of overflow and not many large records. 
 
 Chris
 
 
  From: r...@lynden.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 10:21:16 -0700
  Subject: Re: [U2] RESIZE - dynamic files
  
  (record + id / 4096 or 2048)
  
  You need to factor in overhead  the split factor:   (records + ids) * 1.1 
  * 1.25  / 4096(for 80%) 
  
  If you use a 20% merge factor and a 80% split factor, the file will start 
  merging unless you delete 60 percent of your records.  If you use 90% split 
  factor, you will have more overflowed groups.  These numbers refer to the 
  total amount of data in the file, not to any individual group.
  
  For records of the size that you have, I do not see any advantage to using 
  a larger, 4096, group size. You will end up with twice the number of 
  records per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
  
  -Rick
  
  -Original Message-
  From: u2-users-boun...@listserver.u2ug.org 
  [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
  Sent: Tuesday, July 03, 2012 9:48 AM
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing

Re: [U2] RESIZE - dynamic files

The actual load is 70% on your file. The split.load of 90 was set after the 
file was loaded. If you leave it at that value, and add another 100,000 
records, your modulus will not grow, but the number of overflowed groups will. 

Perhaps you need to look at is as 80% not overflowed.  Despite the output, I 
doubt that any of those overflows are that bad. 




-Rick

On Jul 3, 2012, at 1:23 PM, Chris Austin cjausti...@hotmail.com wrote:

 
 I guess what I need to know is what's an acceptable % of overflow for a 
 dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
 the calculated min modulus)
 I'm still left with ~ 20% of overflow (see below). Is 20% overflow considered 
 acceptable on average or should I keep tinkering with it to reach a lower 
 overflow %?
 
 Correct me if I'm wrong but it seems the goal here is to REDUCE the overflow 
 % while not creating too many modulus (groups).
 
 Chris
 
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
21092 overflowed, 1452 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 70% (actual)
 Total size .   522260480 bytes
 Total size of record data ..   287400239 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   213343528 bytes
 Total space for records    522252288 bytes
 
 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 13:10:43 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 The split load is not affecting anything here, since it is more than the 
 actual load.  What your overflow suggests is that you lower the split.load 
 value to 70$% or below.  You could go ahead and set the merge.load to an 
 arbitrarily low number (1), and it will probably never do a merge, which 
 would be the same as specifying a minimum.modulus equal to as large as it 
 ever gets.  The exception to this is during file creation  clear.file,  
 when the minimum.modulus value determines the initial disk allocation.  
 Short of going to an arbitrarily large minimum.modulus, and a very low 
 split.load, you are going to have some overflow (unless you have sequential 
 keys  like sized records).
 
 -Rick
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 12:54 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Using the formula below, and changing the split to 90% I get the following:
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
22249 overflowed, 1764 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 72% (actual)
 Total size .   519921664 bytes
 Total size of record data ..   287400591 bytes
 Total size of record IDs ...   21508497 bytes
 Unused space ...   211004384 bytes
 Total space for records    519913472 bytes
 
 How does this look in terms of performance?
 
 My Actual load went down 8% as well as some overflow but it looks like my 
 load % is still high at 72% I'm wondering if I should raise the 
 MINIMUM.MODULUS even more
 since I still have a decent amount of overflow and not many large records.
 
 Chris
 
 
 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 10:21:16 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 (record + id / 4096 or 2048)
 
 You need to factor in overhead  the split factor:   (records + ids) * 1.1 
 * 1.25  / 4096(for 80%)
 
 If you use a 20% merge factor and a 80% split factor, the file will start 
 merging unless you delete 60 percent of your records.  If you use 90% split 
 factor, you will have more overflowed groups.  These numbers refer to the 
 total amount of data in the file, not to any individual group.
 
 For records of the size that you have, I do not see any advantage to using 
 a larger, 4096, group size. You will end up with twice the number of 
 records per group vs 2048 (~ 13 vs ~ 7 ), and a little slower keyed access.
 
 -Rick
 
 -Original Message-
 From:

Re: [U2] RESIZE - dynamic files

2012-07-03 Thread Dan Fitzgerald


One rule of thumb is to make sure that you have an average of 10 or less items 
in each group. Going by that, you'd want a minimum mod of 130k or more. I've 
also noticed that files approach the sweet spot for minimizing overflow 
without having excessive empty groups when the total size is pretty nearly 
twice the data size.
 
The goal can vary according to your situation. I'm personally not all that 
afraid of making the modulus a little too large, as overflow is a pretty bad 
performance hit (overflow means at least two disk reads to retrieve your data, 
badly means at least 2 extra disk reads, and I've seen files where that was 
thousands (this file isn't that bad, but 20% of your data is forcing at least 
one extra disk read). Empty groups contribute to overhead on a sequential 
search, so you'd want to consider how often you do a sequential search on a 
file - usually, that's a pretty inefficient way to retrieve data, but, again, 
your mileage may vary. 
 
To me, 20% is too much overflow, and 114 empty groups is trivial; much less 
than 0.2%. I'd be tempted to go to 23 as a minimum Mod, just to see what it 
looks like there. That'll give you an average of 6 records per group, not 
unreasonably shallow, and it's likely to be a while before you have to resize 
again.
 
 From: cjausti...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 15:23:23 -0500
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 I guess what I need to know is what's an acceptable % of overflow for a 
 dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
 the calculated min modulus)
 I'm still left with ~ 20% of overflow (see below). Is 20% overflow considered 
 acceptable on average or should I keep tinkering with it to reach a lower 
 overflow %?
 
 Correct me if I'm wrong but it seems the goal here is to REDUCE the overflow 
 % while not creating too many modulus (groups). 
 
 Chris
 
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
 21092 overflowed, 1452 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 70% (actual)
 Total size .   522260480 bytes
 Total size of record data ..   287400239 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   213343528 bytes
 Total space for records    522252288 bytes
 
  From: r...@lynden.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 13:10:43 -0700
  Subject: Re: [U2] RESIZE - dynamic files
  
  The split load is not affecting anything here, since it is more than the 
  actual load.  What your overflow suggests is that you lower the split.load 
  value to 70$% or below.  You could go ahead and set the merge.load to an 
  arbitrarily low number (1), and it will probably never do a merge, which 
  would be the same as specifying a minimum.modulus equal to as large as it 
  ever gets.  The exception to this is during file creation  clear.file,  
  when the minimum.modulus value determines the initial disk allocation.  
  Short of going to an arbitrarily large minimum.modulus, and a very low 
  split.load, you are going to have some overflow (unless you have sequential 
  keys  like sized records).
  
  -Rick
  
  -Original Message-
  From: u2-users-boun...@listserver.u2ug.org 
  [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
  Sent: Tuesday, July 03, 2012 12:54 PM
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  Using the formula below, and changing the split to 90% I get the following:
  
  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    103889 current ( minimum 103889, 114 empty,
  22249 overflowed, 1764 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 72% (actual)
  Total size .   519921664 bytes
  Total size of record data ..   287400591 bytes
  Total size of record IDs ...   21508497 bytes
  Unused space ...   211004384 bytes
  Total space for records    519913472 bytes
  
  How does this look in terms of performance? 
  
  My Actual load went down 8% as well as some

Re: [U2] Is there any way to tell if I am in an EXECUTE level and CAPTURING is turned on?

2012-07-03 Thread Keith Johnson [DATACOM]

I just noticed this in the help for QM's SYSTEM()

1000 Returns 1 if EXECUTE CAPTURING is in effect, 0 otherwise

Checked if this was an undocumented feature that Martin had reproduced; but no, 
it doesn't work in Universe.

Regards, Keith



___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files


Dan,

I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
Actual Load has really gone down (as well as overflow). See below for the 
results:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
3957 overflowed, 207 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 37% (actual)
Total size .   836235264 bytes
Total size of record data ..   287394719 bytes
Total size of record IDs ...   21508521 bytes
Unused space ...   527323832 bytes
Total space for records    836227072 bytes

My overflow is now @ 2% 
My Load is @ 37% (actual)

granted my empty groups are now up to almost 3% but I hope that won't be a big 
factor. How does this look?

Chris


 From: dangf...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 16:57:34 -0400
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 One rule of thumb is to make sure that you have an average of 10 or less 
 items in each group. Going by that, you'd want a minimum mod of 130k or more. 
 I've also noticed that files approach the sweet spot for minimizing 
 overflow without having excessive empty groups when the total size is pretty 
 nearly twice the data size.
  
 The goal can vary according to your situation. I'm personally not all that 
 afraid of making the modulus a little too large, as overflow is a pretty bad 
 performance hit (overflow means at least two disk reads to retrieve your 
 data, badly means at least 2 extra disk reads, and I've seen files where 
 that was thousands (this file isn't that bad, but 20% of your data is forcing 
 at least one extra disk read). Empty groups contribute to overhead on a 
 sequential search, so you'd want to consider how often you do a sequential 
 search on a file - usually, that's a pretty inefficient way to retrieve data, 
 but, again, your mileage may vary. 
  
 To me, 20% is too much overflow, and 114 empty groups is trivial; much less 
 than 0.2%. I'd be tempted to go to 23 as a minimum Mod, just to see what 
 it looks like there. That'll give you an average of 6 records per group, not 
 unreasonably shallow, and it's likely to be a while before you have to resize 
 again.
  
  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 15:23:23 -0500
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  I guess what I need to know is what's an acceptable % of overflow for a 
  dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
  the calculated min modulus)
  I'm still left with ~ 20% of overflow (see below). Is 20% overflow 
  considered acceptable on average or should I keep tinkering with it to 
  reach a lower overflow %?
  
  Correct me if I'm wrong but it seems the goal here is to REDUCE the 
  overflow % while not creating too many modulus (groups). 
  
  Chris
  
  
  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
  21092 overflowed, 1452 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 70% (actual)
  Total size .   522260480 bytes
  Total size of record data ..   287400239 bytes
  Total size of record IDs ...   21508521 bytes
  Unused space ...   213343528 bytes
  Total space for records    522252288 bytes
  
   From: r...@lynden.com
   To: u2-users@listserver.u2ug.org
   Date: Tue, 3 Jul 2012 13:10:43 -0700
   Subject: Re: [U2] RESIZE - dynamic files
   
   The split load is not affecting anything here, since it is more than the 
   actual load.  What your overflow suggests is that you lower the 
   split.load value to 70$% or below.  You could go ahead and set the 
   merge.load to an arbitrarily low number (1), and it will probably never 
   do a merge, which would be the same as specifying a minimum.modulus equal 
   to as large as it ever gets.  The exception to this is during file 
   creation  clear.file,  when the minimum.modulus value determines the 
   initial disk allocation.  Short of going to an arbitrarily large 
   minimum.modulus, and a very low split.load, you are going to have some 
   overflow

Re: [U2] RESIZE - dynamic files

But the total size of your file is up 60%.  Reading in 60% more records in a 
full select of the file is going to be much slower than a few more overflows.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 2:15 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files

Dan,

I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
Actual Load has really gone down (as well as overflow). See below for the 
results:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
3957 overflowed, 207 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 37% (actual)
Total size .   836235264 bytes
Total size of record data ..   287394719 bytes
Total size of record IDs ...   21508521 bytes
Unused space ...   527323832 bytes
Total space for records    836227072 bytes

My overflow is now @ 2%
My Load is @ 37% (actual)

granted my empty groups are now up to almost 3% but I hope that won't be a big 
factor. How does this look?

Chris

 From: dangf...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 16:57:34 -0400
 Subject: Re: [U2] RESIZE - dynamic files

 One rule of thumb is to make sure that you have an average of 10 or less 
 items in each group. Going by that, you'd want a minimum mod of 130k or more. 
 I've also noticed that files approach the sweet spot for minimizing 
 overflow without having excessive empty groups when the total size is pretty 
 nearly twice the data size.

 The goal can vary according to your situation. I'm personally not all that 
 afraid of making the modulus a little too large, as overflow is a pretty bad 
 performance hit (overflow means at least two disk reads to retrieve your 
 data, badly means at least 2 extra disk reads, and I've seen files where 
 that was thousands (this file isn't that bad, but 20% of your data is forcing 
 at least one extra disk read). Empty groups contribute to overhead on a 
 sequential search, so you'd want to consider how often you do a sequential 
 search on a file - usually, that's a pretty inefficient way to retrieve data, 
 but, again, your mileage may vary.

 To me, 20% is too much overflow, and 114 empty groups is trivial; much less 
 than 0.2%. I'd be tempted to go to 23 as a minimum Mod, just to see what 
 it looks like there. That'll give you an average of 6 records per group, not 
 unreasonably shallow, and it's likely to be a while before you have to resize 
 again.

  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 15:23:23 -0500
  Subject: Re: [U2] RESIZE - dynamic files

  I guess what I need to know is what's an acceptable % of overflow for a 
  dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
  the calculated min modulus)
  I'm still left with ~ 20% of overflow (see below). Is 20% overflow 
  considered acceptable on average or should I keep tinkering with it to 
  reach a lower overflow %?

  Correct me if I'm wrong but it seems the goal here is to REDUCE the 
  overflow % while not creating too many modulus (groups).

  Chris

  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
  21092 overflowed, 1452 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 70% (actual)
  Total size .   522260480 bytes
  Total size of record data ..   287400239 bytes
  Total size of record IDs ...   21508521 bytes
  Unused space ...   213343528 bytes
  Total space for records    522252288 bytes

   From: r...@lynden.com
   To: u2-users@listserver.u2ug.org
   Date: Tue, 3 Jul 2012 13:10:43 -0700
   Subject: Re: [U2] RESIZE - dynamic files

   The split load is not affecting anything here, since it is more than the 
   actual load.  What your overflow suggests is that you lower the 
   split.load value to 70$% or below.  You could go ahead and set the 
   merge.load to an arbitrarily low number (1), and it will

Re: [U2] * GMX Spamverdacht * UniVerse LIST statement question [not-secure]

2012-07-03 Thread Kate Stanton

You can do just about anything by calling a subroutine from an I-type
dictionary item.
On 3 July 2012 05:41, Mecki Foerthmann mec...@gmx.net wrote:

Since we don't have outer joins I would build a work file starting with
selecting all employees, then select the calls and merge them together in a
loop. How you do it depends on how you want to present the data.
If you're happy with showing the calls as multi values you can build the
records that way and if you like one row per call you should probably have
one row per call.
Then report on the work file.
That is the easiest way I can see to include employees who didn't make any
calls.
I can also think of a few approaches involving stored parameters and
dictionaries or a dictionary in the employee file that builds the call info
in a subroutine as multivalues using an index by employee on the calls file.
In any case you will most likely need to base your report on the employees
file not the calls.

On 02/07/2012 14:53, Hennessey, Mark F. wrote:

I need to do a UniVerse LIST statement that would only populate a column
if the contents met certain criteria.

For example, suppose we have a file with details of telephone usage and
that 3 associated mulitvalued fields contain date call was made, duration
and if the call was a toll call. Is it possible to limit the output of the
date call made and associated columns to a date range without that being
a select criteria? If I were to do something like:

LIST CALLS EMP.NAME EMP.LOCATION WITH DATE.CALL GE 2012-06-01 AND WITH
DATE.CALL LE 2012-06-30 DURATION TOLL WITH @ID EQ '123456'

I would get zero record if employee 123456 did not make any calls in
June. What I would like to see is the employer name and location returned
with the date, duration and toll columns empty. I'm trying to do this in a
LIST statement as it will be run by U2 Web Services (and for the time being
a subroutine is off the table...)

Any advice, or an authoritative NO, It can not be done would be greatly
appreciated.

Mark Hennessey
State of Connecticut
Department of Social Services
Information Technology Services
Child Support Systems
Voice: 860-424-5261
Fax: 860-424-4813

CONFIDENTIAL INFORMATION: The information contained in this e-mail may be
confidential and protected from general disclosure. If the recipient or
reader of this e-mail is not the intended recipient or a person responsible
to receive this e-mail for the intended recipient, please do not
disseminate, distribute or copy it. If you received this e-mail in error,
please notify the sender by replying to this message and delete this e-mail
immediately. We will take immediate and appropriate action to see to it
that this mistake is corrected.[*LD*]
__**_
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/**mailman/listinfo/u2-usershttp://listserver.u2ug.org/mailman/listinfo/u2-users

__**_
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/**mailman/listinfo/u2-usershttp://listserver.u2ug.org/mailman/listinfo/u2-users

--
Kate Stanton
Walstan Systems Ltd
4 Kelmarna Ave, Herne Bay, Auckland 1011, New Zealand
Phone: + 64 9 360 5310 Mobile: + 64 21 400 486
Email: k...@walstan.com
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

I should have said 60% more disk records, to be clear.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Rick Nuckolls
Sent: Tuesday, July 03, 2012 2:24 PM
To: 'U2 Users List'
Subject: Re: [U2] RESIZE - dynamic files

But the total size of your file is up 60%.  Reading in 60% more records in a 
full select of the file is going to be much slower than a few more overflows.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 2:15 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files

Dan,

I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
Actual Load has really gone down (as well as overflow). See below for the 
results:

File name ..   GENACCTRN_POSTED
Pathname ...   GENACCTRN_POSTED
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
3957 overflowed, 207 badly )
Number of records ..   1290469
Large record size ..   3267 bytes
Number of large records    180
Group size .   4096 bytes
Load factors ...   90% (split), 50% (merge) and 37% (actual)
Total size .   836235264 bytes
Total size of record data ..   287394719 bytes
Total size of record IDs ...   21508521 bytes
Unused space ...   527323832 bytes
Total space for records    836227072 bytes

My overflow is now @ 2%
My Load is @ 37% (actual)

granted my empty groups are now up to almost 3% but I hope that won't be a big 
factor. How does this look?

Chris

 From: dangf...@hotmail.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 16:57:34 -0400
 Subject: Re: [U2] RESIZE - dynamic files

 One rule of thumb is to make sure that you have an average of 10 or less 
 items in each group. Going by that, you'd want a minimum mod of 130k or more. 
 I've also noticed that files approach the sweet spot for minimizing 
 overflow without having excessive empty groups when the total size is pretty 
 nearly twice the data size.

 The goal can vary according to your situation. I'm personally not all that 
 afraid of making the modulus a little too large, as overflow is a pretty bad 
 performance hit (overflow means at least two disk reads to retrieve your 
 data, badly means at least 2 extra disk reads, and I've seen files where 
 that was thousands (this file isn't that bad, but 20% of your data is forcing 
 at least one extra disk read). Empty groups contribute to overhead on a 
 sequential search, so you'd want to consider how often you do a sequential 
 search on a file - usually, that's a pretty inefficient way to retrieve data, 
 but, again, your mileage may vary.

 To me, 20% is too much overflow, and 114 empty groups is trivial; much less 
 than 0.2%. I'd be tempted to go to 23 as a minimum Mod, just to see what 
 it looks like there. That'll give you an average of 6 records per group, not 
 unreasonably shallow, and it's likely to be a while before you have to resize 
 again.

  From: cjausti...@hotmail.com
  To: u2-users@listserver.u2ug.org
  Date: Tue, 3 Jul 2012 15:23:23 -0500
  Subject: Re: [U2] RESIZE - dynamic files

  I guess what I need to know is what's an acceptable % of overflow for a 
  dynamic file? For example, when I change the SPLIT LOAD to 90% (while using 
  the calculated min modulus)
  I'm still left with ~ 20% of overflow (see below). Is 20% overflow 
  considered acceptable on average or should I keep tinkering with it to 
  reach a lower overflow %?

  Correct me if I'm wrong but it seems the goal here is to REDUCE the 
  overflow % while not creating too many modulus (groups).

  Chris

  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    105715 current ( minimum 103889, 114 empty,
  21092 overflowed, 1452 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 70% (actual)
  Total size .   522260480 bytes
  Total size of record data ..   287400239 bytes
  Total size of record IDs ...   21508521 bytes
  Unused space ...   213343528 bytes
  Total space for records    522252288 bytes

   From: r...@lynden.com
   To: u2-users@listserver.u2ug.org
   Date: Tue, 3 Jul 2012 13:10:43 -0700
   Subject: Re: [U2]

Re: [U2] RESIZE - dynamic files


This is why I'm confused.. Is the goal here to reduce 'overflow' or to 
keep the 'Total size' of the disk down? If the goal is to keep the total
 disk size down then it would appear
you would want your actual load % a lot higher than 37%.. and then ignore 
'some' of the overflow..

Chris


 But the total size of your file is up 60%.  Reading in 60% more records in a 
 full select of the file is going to be much slower than a few more overflows.
 
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:15 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Dan,
 
 I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
 Actual Load has really gone down (as well as overflow). See below for the 
 results:
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
 3957 overflowed, 207 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 37% (actual)
 Total size .   836235264 bytes
 Total size of record data ..   287394719 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   527323832 bytes
 Total space for records    836227072 bytes
 
 My overflow is now @ 2%
 My Load is @ 37% (actual)
 
 granted my empty groups are now up to almost 3% but I hope that won't be a 
 big factor. How does this look?
 
 Chris

  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

2012-07-03 Thread Wjhonson

Disks get bigger much faster than the rate they get faster.
So the overflow is the thing to minimize.

-Original Message-
From: Chris Austin cjausti...@hotmail.com
To: u2-users u2-users@listserver.u2ug.org
Sent: Tue, Jul 3, 2012 2:38 pm
Subject: Re: [U2] RESIZE - dynamic files

his is why I'm confused.. Is the goal here to reduce 'overflow' or to 
eep the 'Total size' of the disk down? If the goal is to keep the total
disk size down then it would appear
ou would want your actual load % a lot higher than 37%.. and then ignore 'some' 
f the overflow..
Chris

 But the total size of your file is up 60%.  Reading in 60% more records in a 
ull select of the file is going to be much slower than a few more overflows.

 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] 
n Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:15 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files

 Dan,

 I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
ctual Load has really gone down (as well as overflow). See below for the 
esults:

 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
 3957 overflowed, 207 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 37% (actual)
 Total size .   836235264 bytes
 Total size of record data ..   287394719 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   527323832 bytes
 Total space for records    836227072 bytes

 My overflow is now @ 2%
 My Load is @ 37% (actual)

 granted my empty groups are now up to almost 3% but I hope that won't be a big 
actor. How does this look?

 Chris

__
2-Users mailing list
2-us...@listserver.u2ug.org
ttp://listserver.u2ug.org/mailman/listinfo/u2-users

___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

37% is a very low load.  Reading disk records takes much longer than parsing 
the records out of a disk record.  With variable record size and moderately 
poor hashing, overflow is inevitable.  So, do you want 80,000 extra groups, or 
20,000 overflow buffers? I would go with the smaller number.  But for the love 
of Knuth, do not set your split.load to 90% unless you have a perfectly hashed 
file with uniformly sized records.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 2:38 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


This is why I'm confused.. Is the goal here to reduce 'overflow' or to 
keep the 'Total size' of the disk down? If the goal is to keep the total
 disk size down then it would appear
you would want your actual load % a lot higher than 37%.. and then ignore 
'some' of the overflow..

Chris


 But the total size of your file is up 60%.  Reading in 60% more records in a 
 full select of the file is going to be much slower than a few more overflows.
 
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:15 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 Dan,
 
 I changed the MINIMUM.MODULUS to the value of 23 as you suggested and my 
 Actual Load has really gone down (as well as overflow). See below for the 
 results:
 
 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
 3957 overflowed, 207 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 37% (actual)
 Total size .   836235264 bytes
 Total size of record data ..   287394719 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   527323832 bytes
 Total space for records    836227072 bytes
 
 My overflow is now @ 2%
 My Load is @ 37% (actual)
 
 granted my empty groups are now up to almost 3% but I hope that won't be a 
 big factor. How does this look?
 
 Chris

  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

2012-07-03 Thread Susan Lynch

Chris,

This is why file-sizing is something that requires careful thought.  As
some of the other responders have indicated, sometimes you want to keep
overflow to a minimum (because accessing individual records that are in
overflow takes extra disk reads, which slow down your system, and adding
new records to a group that is already in overflow will inevitably be
slower than adding a new record to a group which is not in overflow), and
sometimes you don't (eg if you have a file that is primarily read in a
sequential fashion where you do a Basic SELECT, and then loop through the
file reading every single record).   Because most of the files that I have
supported in my career have been read and written primarily as
single-record reads, I have always chosen to minimize overflow as my
default criteria, and only sized things for sequential reads when the file
is rarely written, rarely read as anything but a 'read them all in no
particular order' fashion, and that happens rarely in my experience.
However, as other responders have written, 'your mileage may vary'!

Look at how the file is used.  Look at what resources you have.  Then
decide...


Susan M. Lynch
F. W. Davison  Company, Inc.
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: 07/03/2012 5:38 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


This is why I'm confused.. Is the goal here to reduce 'overflow' or to
keep the 'Total size' of the disk down? If the goal is to keep the total
 disk size down then it would appear
you would want your actual load % a lot higher than 37%.. and then ignore
'some' of the overflow..

Chris


 But the total size of your file is up 60%.  Reading in 60% more records
in a full select of the file is going to be much slower than a few more
overflows.


 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:15 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files


 Dan,

 I changed the MINIMUM.MODULUS to the value of 23 as you suggested
and my Actual Load has really gone down (as well as overflow). See below
for the results:

 File name ..   GENACCTRN_POSTED
 Pathname ...   GENACCTRN_POSTED
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    23 current ( minimum 23, 5263
empty,
 3957 overflowed, 207 badly )
 Number of records ..   1290469
 Large record size ..   3267 bytes
 Number of large records    180
 Group size .   4096 bytes
 Load factors ...   90% (split), 50% (merge) and 37% (actual)
 Total size .   836235264 bytes
 Total size of record data ..   287394719 bytes
 Total size of record IDs ...   21508521 bytes
 Unused space ...   527323832 bytes
 Total space for records    836227072 bytes

 My overflow is now @ 2%
 My Load is @ 37% (actual)

 granted my empty groups are now up to almost 3% but I hope that won't be
a big factor. How does this look?

 Chris


___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files


I set the split load based on what Dan suggested:

I'd take the merge down a little, to maybe 30% or even less, and maybe knock 
the split up a bit - say, 90% - to cut down on the splitting.

I thought this would cut down on splitting. Is there a certain formula, or way 
to calculate the split.load? What should my SPLIT.LOAD be around,
and how do you come up with that %?

Chris

 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 14:45:28 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 37% is a very low load.  Reading disk records takes much longer than parsing 
 the records out of a disk record.  With variable record size and moderately 
 poor hashing, overflow is inevitable.  So, do you want 80,000 extra groups, 
 or 20,000 overflow buffers? I would go with the smaller number.  But for the 
 love of Knuth, do not set your split.load to 90% unless you have a perfectly 
 hashed file with uniformly sized records.
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:38 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 This is why I'm confused.. Is the goal here to reduce 'overflow' or to 
 keep the 'Total size' of the disk down? If the goal is to keep the total
  disk size down then it would appear
 you would want your actual load % a lot higher than 37%.. and then ignore 
 'some' of the overflow..
 
 Chris
 
 
  But the total size of your file is up 60%.  Reading in 60% more records in 
  a full select of the file is going to be much slower than a few more 
  overflows.
  
  
  -Original Message-
  From: u2-users-boun...@listserver.u2ug.org 
  [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
  Sent: Tuesday, July 03, 2012 2:15 PM
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  Dan,
  
  I changed the MINIMUM.MODULUS to the value of 23 as you suggested and 
  my Actual Load has really gone down (as well as overflow). See below for 
  the results:
  
  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
  3957 overflowed, 207 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 37% (actual)
  Total size .   836235264 bytes
  Total size of record data ..   287394719 bytes
  Total size of record IDs ...   21508521 bytes
  Unused space ...   527323832 bytes
  Total space for records    836227072 bytes
  
  My overflow is now @ 2%
  My Load is @ 37% (actual)
  
  granted my empty groups are now up to almost 3% but I hope that won't be a 
  big factor. How does this look?
  
  Chris
 
 
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

[U2] Account.File.Stats ALL

2012-07-03 Thread Wjhonson


When you execute the Account.File.Stats command in Universe, with the ALL 
directive, the code executes the following command to make it's selection

SSELECT VOC WITH TYPE = F AND F2 NOT.MATCHING 'I_'... AND F2 NOT.MATCHING 
'../'... AND F2 NOT.MATCHING '..\'... AND F2 NOT.MATCHING '/'... AND F2 
NOT.MATCHING '\'... AND F2 NOT.MATCHING 1A:'\'... AND F2 # . AND F2 # 


___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

Unless the minimum modulus is configured high enough to artificially lower the 
actual load, the actual load will rise to the designated split.load as the file 
grows. The split.load indicates nothing about the specific load of any given 
group; so if it is set to 90%, then on average, each group will be 90% full, 
and adding a (400byte) record to a group will send it into overflow, but since 
400 bytes is a trivial percentage of your overall file load, many groups will 
be overflowed before the total load factor exceeds 90%.  

Okay, there is a slight distortion with the numbers there, but the idea is that 
all buckets are not loaded equally, so if the average is almost full the 
reality is many overflowed.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Tuesday, July 03, 2012 2:52 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files


I set the split load based on what Dan suggested:

I'd take the merge down a little, to maybe 30% or even less, and maybe knock 
the split up a bit - say, 90% - to cut down on the splitting.

I thought this would cut down on splitting. Is there a certain formula, or way 
to calculate the split.load? What should my SPLIT.LOAD be around,
and how do you come up with that %?

Chris

 From: r...@lynden.com
 To: u2-users@listserver.u2ug.org
 Date: Tue, 3 Jul 2012 14:45:28 -0700
 Subject: Re: [U2] RESIZE - dynamic files
 
 37% is a very low load.  Reading disk records takes much longer than parsing 
 the records out of a disk record.  With variable record size and moderately 
 poor hashing, overflow is inevitable.  So, do you want 80,000 extra groups, 
 or 20,000 overflow buffers? I would go with the smaller number.  But for the 
 love of Knuth, do not set your split.load to 90% unless you have a perfectly 
 hashed file with uniformly sized records.
 
 -Original Message-
 From: u2-users-boun...@listserver.u2ug.org 
 [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
 Sent: Tuesday, July 03, 2012 2:38 PM
 To: u2-users@listserver.u2ug.org
 Subject: Re: [U2] RESIZE - dynamic files
 
 
 This is why I'm confused.. Is the goal here to reduce 'overflow' or to 
 keep the 'Total size' of the disk down? If the goal is to keep the total
  disk size down then it would appear
 you would want your actual load % a lot higher than 37%.. and then ignore 
 'some' of the overflow..
 
 Chris
 
 
  But the total size of your file is up 60%.  Reading in 60% more records in 
  a full select of the file is going to be much slower than a few more 
  overflows.
  
  
  -Original Message-
  From: u2-users-boun...@listserver.u2ug.org 
  [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
  Sent: Tuesday, July 03, 2012 2:15 PM
  To: u2-users@listserver.u2ug.org
  Subject: Re: [U2] RESIZE - dynamic files
  
  
  Dan,
  
  I changed the MINIMUM.MODULUS to the value of 23 as you suggested and 
  my Actual Load has really gone down (as well as overflow). See below for 
  the results:
  
  File name ..   GENACCTRN_POSTED
  Pathname ...   GENACCTRN_POSTED
  File type ..   DYNAMIC
  File style and revision    32BIT Revision 12
  Hashing Algorithm ..   GENERAL
  No. of groups (modulus)    23 current ( minimum 23, 5263 empty,
  3957 overflowed, 207 badly )
  Number of records ..   1290469
  Large record size ..   3267 bytes
  Number of large records    180
  Group size .   4096 bytes
  Load factors ...   90% (split), 50% (merge) and 37% (actual)
  Total size .   836235264 bytes
  Total size of record data ..   287394719 bytes
  Total size of record IDs ...   21508521 bytes
  Unused space ...   527323832 bytes
  Total space for records    836227072 bytes
  
  My overflow is now @ 2%
  My Load is @ 37% (actual)
  
  granted my empty groups are now up to almost 3% but I hope that won't be a 
  big factor. How does this look?
  
  Chris
 
 
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
 ___
 U2-Users mailing list
 U2-Users@listserver.u2ug.org
 http://listserver.u2ug.org/mailman/listinfo/u2-users
  
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

2012-07-03 Thread Charles Stevenson


Chris,
Let's back way up.   I take it your original question is a general one,  
not specific to one poorly performing problematic file.  Is that right?


If so, generally speaking, you just don't get a lot out of fine-tuning 
dynamic files.
Tweaking the default parameters doesn't usually make a whole lot of 
difference.

Several people have said something similar in this thread.

Other than deciding which hashing algorithm,  I generally use the 
defaults and only tweak things once the file proves problematic, which 
usually means slow I/O.


When a problem erupts, look carefully at how that specific file is used, 
as Susan  others have said.   You might get hold of FitzgeraldLong's 
paper on how dynamic files work.  If you understand the fundamentals, 
you'll understand how to attack your problem file, applying the ideas 
Rick  others have talked about here.


You may go several years without having to resort to that.

Chuck Stevenson


On 7/2/2012 2:22 PM, Chris Austin wrote:

I was wondering if anyone had instructions on RESIZE with a dynamic file? For 
example I have a file called 'TEST_FILE'
with the following:

01 ANALYZE.FILE TEST_FILE
File name ..   TEST_FILE
Pathname ...   TEST_FILE
File type ..   DYNAMIC
File style and revision    32BIT Revision 12
Hashing Algorithm ..   GENERAL
No. of groups (modulus)    83261 current ( minimum 31 )
Large record size ..   3267 bytes
Group size .   4096 bytes
Load factors ...   80% (split), 50% (merge) and 80% (actual)
Total size .   450613248 bytes

How do you calculate what the modulus and separation should be? I can't use 
HASH.HELP on a type 30 file to see the recommended settings
so I was wondering how best you figure out the file RESIZE.

Thanks,

Chris



___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] RESIZE - dynamic files

From the System Description manual:

Important Considerations

Dynamic files are meant to make file management easier for users. The default
parameters are set so that most dynamic files work efficiently. If you decide 
to change
the parameters of a dynamic file, keep the following considerations in mind:

 Use the SEQ.NUM hashing algorithm only when your record IDs are
numeric, sequential, and consecutive. Nonconsecutive numbers should not
be hashed using the SEQ.NUM hashing algorithm.

 Use a group size of 2 only if you expect the average record size to be larger
than 1000 bytes. If your record size is larger than 2000 bytes, consider using
a nonhashed file—type 1 or 19.

 Large record size should generally not be changed. Storing the data of a
large record in the overflow buffer causes that data not to be included in the
split and merge calculations. Also, the extra data length does not slow access
to subsequent records. By choosing a large record size of 0%, all the records
are considered large. In this case, record IDs can be accessed extremely
quickly by commands such as SELECT, but access to the actual data is
much less efficient.

 A small split load causes less data to be stored in each group buffer, 
resulting
in faster access time and less overflow at the expense of requiring extra
memory. A large split load causes more data to be stored in each group
buffer, resulting in better use of memory at the expense of slower access
time and more overflow. A split load of 100% disables splits.

 The gap between merge load and split load should be large enough so that
splits and merges do not occur too frequently. The split and merge processes
take a significant amount of processing time. If you make the merge load too
small, memory usage can be very poor. Also, selection time is increased
when record IDs are distributed in more groups than are needed. A merge
load of 0% disables merges.

 Consider increasing the minimum modulo if you intend to add a lot of initial
data to the file. Much data-entry time can be saved by avoiding the initial
splits that can occur if you enter a lot of initial data. You may want to
readjust this value after

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson
Sent: Tuesday, July 03, 2012 3:34 PM
To: U2 Users List
Subject: Re: [U2] RESIZE - dynamic files

Chris,
Let's back way up.   I take it your original question is a general one,  
not specific to one poorly performing problematic file.  Is that right?

If so, generally speaking, you just don't get a lot out of fine-tuning 
dynamic files.
Tweaking the default parameters doesn't usually make a whole lot of 
difference.
Several people have said something similar in this thread.

Other than deciding which hashing algorithm,  I generally use the 
defaults and only tweak things once the file proves problematic, which 
usually means slow I/O.

When a problem erupts, look carefully at how that specific file is used, 
as Susan  others have said.   You might get hold of FitzgeraldLong's 
paper on how dynamic files work.  If you understand the fundamentals, 
you'll understand how to attack your problem file, applying the ideas 
Rick  others have talked about here.

You may go several years without having to resort to that.

Chuck Stevenson


On 7/2/2012 2:22 PM, Chris Austin wrote:
 I was wondering if anyone had instructions on RESIZE with a dynamic file? For 
 example I have a file called 'TEST_FILE'
 with the following:

 01 ANALYZE.FILE TEST_FILE
 File name ..   TEST_FILE
 Pathname ...   TEST_FILE
 File type ..   DYNAMIC
 File style and revision    32BIT Revision 12
 Hashing Algorithm ..   GENERAL
 No. of groups (modulus)    83261 current ( minimum 31 )
 Large record size ..   3267 bytes
 Group size .   4096 bytes
 Load factors ...   80% (split), 50% (merge) and 80% (actual)
 Total size .   450613248 bytes

 How do you calculate what the modulus and separation should be? I can't use 
 HASH.HELP on a type 30 file to see the recommended settings
 so I was wondering how best you figure out the file RESIZE.

 Thanks,

 Chris


___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] Running XLr8 Tools inside U2 DBTools new Eclipse release

2012-07-03 Thread Brian Whitehorn

I don't have BDT or other Rocket tools installed to confirm, but would 
specifying the same -data workspacepath in the arguments for each achieve 
what you're after ?
(via: 
http://help.eclipse.org/indigo/index.jsp?topic=%2Forg.eclipse.platform.doc.user%2Ftasks%2Frunning_eclipse.htm)

Regards,
Brian.

-Original Message-
From: u2-users-boun...@listserver.u2ug.org 
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch
Sent: Wednesday, 4 July 2012 3:23 AM
To: U2 Users List
Subject: Re: [U2] Running XLr8 Tools inside U2 DBTools  new Eclipse release

Hi Dan and others:

I finished testing all of the Rocket tools.  Turns out the only one that really 
works as a plug-in platform is BDT for U2logic plug-ins.  The other products 
are crippled versions of Eclipse with limited menus by design.

Another big problem is that Rocket's Eclipse products do not share work spaces, 
so if you start BDT your work space is in that directory.  If you start U2Admin 
the work space is in that directory.  Unfortunately this situation is very 
messy for U2logic, a Rocket VAR, to support.  I'm going to have fun explaining 
this to the first client that asks why the programs don't share the server 
names.

Regards,
Doug
www.u2logic.com
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
--
Message  protected by DealerGuard: e-mail anti-virus, anti-spam and content 
filtering.
http://www.pentanasolutions.com

Click here to report this message as spam:
https://login.mailguard.com.au/report/1F4dy0mK0d/4ZdT5DKh528rfiCd3YgdAP/0.204


This email and any attachments to it are confidential.
You must not use, disclose or act on the email if you are not the intended
recipient.  Liability limited by a scheme approved under Professional
Standards Legislation.
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users

Re: [U2] Running XLr8 Tools inside U2 DBTools new Eclipse release