Re: [U2] T30File parameter in Universe 10.3.7
Don't use too many dynamic files myself. However, I would imagine if you... Assuming you are on *nix Assuming uvhome is /usr/uv -Login to uv account MASTER OFF ALL -shutdown uv /usr/uv/bin/uv -admin -stop -changed the parameter in /usr/uv/uvconfig (Just use your favorite text editor) -do a uvregen cd /usr/uv bin/uvregen -Stop uv again to make sure acec segment did not get started /usr/uv/bin/uv -admin -stop -start up uv again /usr/uv/bin/uv -admin -start -verify change took effect bin/uv/smat -t (Look for T30FILE in output) Of course this will kick everyone off. In windows there might be some gui tool to do it. On 7/6/12, Allen Egerton wrote: > The default value for T30Files is 200, and it's described as controlling > the number of dynamic files that can be open on a system at any point in > time. > > Has anyone had practical experience with increasing this limit? > > And yes I'm asking because we just crashed into this wall --- and it > wasn't pretty. > > Thank you. > > -- > Allen Egerton > aeger...@pobox.com > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > -- Sent from my mobile device John Thompson ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
Note - there is a UniBasic LISTUSER() function that returns the detail lines in a dynamic array for easy parsing. Wally Terhune Technical Support Engineer Rocket Software 4600 South Ulster Street, Suite 1100 **Denver, CO 80237 **USA t: +1 720 475 8055 **e: wterh...@rocketsoftware.com **w: rocketsoftware.com/u2 -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: Friday, July 06, 2012 12:50 PM To: U2 Users List Cc: u2-users-boun...@listserver.u2ug.org Subject: Re: [U2] Monitoring connections on Unidata >>I wrote a pgm that executes !listuser, captures the license counts >>from the output (maybe line 4?), rips that data apart and writes the data to a log file (LICENSES). This job runs every 15 >>minutes. >>I then wrote a program that runs every morning and exports all the >>data to a tab-delimited txt file and E-mails it to the IT management. Any data over 90 days is purged (so my file is fairly >>static in size). >>Management then open it in Excel and make pretty pictures with charts and pivot tables. It allows them to track when we are busy, how many licenses we are using, etc. They have found this to be >>very helpful. >>Maybe this would help. >>JRI You bet. Management loves numbers. Is it Unibasic? Thanks. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
What I do to monitor ongoing license usage is run a program via ON.ENTRY that checks the current user count total and writes it to a log record for today's date if it's greater than the current entry. That gives me a high water mark for each day. A program runs during nightly batch processing that sends an email alert if the high water mark is within 5 seats of our total licenses. -John -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: Friday, July 06, 2012 11:41 AM To: U2 Users List Subject: Re: [U2] Monitoring connections on Unidata >>John Hester >>That sounds like the correct way to determine total license count. In >>UV at least, there is also a more official way to do it via a built-in >>subroutine called !GET.USER.COUNTS. I don't know if it exists in UD. >>I have a simple program I can run at TCL that calls it and reports the >>numbers: >>0001: CALL !GET.USER.COUNTS (uv.users, max.uv.users, os.users) >>0002: PRINT "Max UniVerse users: ":max.uv.users >>0003: PRINT "Current UniVerse users: ":uv.users >>0004: PRINT "OS users : ":os.users Thanks. I used the code you shared and some of my own to show the Users, RPC connections and the sum. This will give me a tool to watch the number of connections and compare to the occurrences of problems. Naturally, since it is late on Friday afternoon, things are quieting down. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
Yes it is. JRI -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: Friday, July 06, 2012 2:50 PM To: U2 Users List Cc: u2-users-boun...@listserver.u2ug.org Subject: Re: [U2] Monitoring connections on Unidata >>I wrote a pgm that executes !listuser, captures the license counts >>from the output (maybe line 4?), rips that data apart and writes the data to a log file (LICENSES). This job runs every 15 >>minutes. >>I then wrote a program that runs every morning and exports all the >>data to a tab-delimited txt file and E-mails it to the IT management. Any data over 90 days is purged (so my file is fairly >>static in size). >>Management then open it in Excel and make pretty pictures with charts and pivot tables. It allows them to track when we are busy, how many licenses we are using, etc. They have found this to be >>very helpful. >>Maybe this would help. >>JRI You bet. Management loves numbers. Is it Unibasic? Thanks. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
>>Symeon. >>Listuser shows rpc connections. The tty shows as udcs, if it is using a >>uniobjects.net pooled connection, the type is pooled, otherwise it is udt Is total license usage, the listuser plus the udcs connections? Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
>>I wrote a pgm that executes !listuser, captures the license counts from the output (maybe line 4?), rips that data apart and writes the data to a log file (LICENSES). This job runs every 15 >>minutes. >>I then wrote a program that runs every morning and exports all the data to a tab-delimited txt file and E-mails it to the IT management. Any data over 90 days is purged (so my file is fairly >>static in size). >>Management then open it in Excel and make pretty pictures with charts and pivot tables. It allows them to track when we are busy, how many licenses we are using, etc. They have found this to be >>very helpful. >>Maybe this would help. >>JRI You bet. Management loves numbers. Is it Unibasic? Thanks. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
Listuser shows rpc connections. The tty shows as udcs, if it is using a uniobjects.net pooled connection, the type is pooled, otherwise it is udt -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: 06 July 2012 16:34 To: U2 Users List Cc: u2-users-boun...@listserver.u2ug.org Subject: Re: [U2] Monitoring connections on Unidata What about connections coming from client applications? UniCall, UniObjects, etc. Do they use up licenses also? Charles Shaffer Senior Analyst NTN-Bower Corporation From: "Symeon Breen" To: "'U2 Users List'" , Date: 07/06/2012 10:18 AM Subject:Re: [U2] Monitoring connections on Unidata Sent by:u2-users-boun...@listserver.u2ug.org listuser ? -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: 06 July 2012 14:54 To: U2 Users List Subject: [U2] Monitoring connections on Unidata Is there a way to monitor the number of RPC connections being used at any given time on Unidata 7.1/Redhat 3? We are not using device licensing. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.2195 / Virus Database: 2437/5112 - Release Date: 07/05/12 ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.2195 / Virus Database: 2437/5112 - Release Date: 07/05/12 ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
I wrote a pgm that executes !listuser, captures the license counts from the
output (maybe line 4?), rips that data apart and writes the data to a log file
(LICENSES). This job runs every 15 minutes.
I then wrote a program that runs every morning and exports all the data to a
tab-delimited txt file and E-mails it to the IT management. Any data over 90
days is purged (so my file is fairly static in size).
Management then open it in Excel and make pretty pictures with charts and pivot
tables. It allows them to track when we are busy, how many licenses we are
using, etc. They have found this to be very helpful.
Maybe this would help.
JRI
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of John Hester
Sent: Friday, July 06, 2012 2:26 PM
To: U2 Users List
Subject: Re: [U2] Monitoring connections on Unidata
That sounds like the correct way to determine total license count. In UV at
least, there is also a more official way to do it via a built-in subroutine
called !GET.USER.COUNTS. I don't know if it exists in UD. I have a simple
program I can run at TCL that calls it and reports the
numbers:
0001: CALL !GET.USER.COUNTS (uv.users, max.uv.users, os.users)
0002: PRINT "Max UniVerse users: ":max.uv.users
0003: PRINT "Current UniVerse users: ":uv.users
0004: PRINT "OS users : ":os.users
-John
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of
charles_shaf...@ntn-bower.com
Sent: Friday, July 06, 2012 11:08 AM
To: U2 Users List
Subject: Re: [U2] Monitoring connections on Unidata
>> On UV, counting instances of uvapi_slave tells me how many unirpc
>> connections are active via UOJ. It doesn't include JDBC connections,
>> though. JDBC connections spawn a process called uvserver. If
capturing
>> every type of unirpc connection is important, you could determine the
>> PID of unirpcd and count its immediate children. I think this should
>> work:
>> PID=`ps -el | grep unirpcd | awk '{print $4}'` COUNT=`ps -el | grep
>> $PID | awk '{print $5}' | grep $PID | wc -l` echo $COUNT
>> -John
Thanks. That works great.
The problem I am having is that our MRP application does a lot of RPC calls to
our Unidata server. We have been having a lot of random problems and I suspect
that we are running out of connections. I am looking for a way to show that
for sure. I am recommending beefing up our licenses and need to show some
justification for the financial expense.
Does the sum of the number of users plus the number of RPC connections give me
the number of licenses in use at the moment?
It seems that way. Right now there are 56 users listed in listuser.
There are 31 RPC connections. This is a total of 87. We have 96 licenses.
And right now everybody seems to be happy. I expect that when problems start
occurring, I will check and find that the total of the list user count and RPC
connections will be greater than 96.
Does that sound right?
Charles Shaffer
Senior Analyst
NTN-Bower Corporation
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
>>John Hester >>That sounds like the correct way to determine total license count. In >>UV at least, there is also a more official way to do it via a built-in >>subroutine called !GET.USER.COUNTS. I don't know if it exists in UD. I >>have a simple program I can run at TCL that calls it and reports the >>numbers: >>0001: CALL !GET.USER.COUNTS (uv.users, max.uv.users, os.users) >>0002: PRINT "Max UniVerse users: ":max.uv.users >>0003: PRINT "Current UniVerse users: ":uv.users >>0004: PRINT "OS users : ":os.users Thanks. I used the code you shared and some of my own to show the Users, RPC connections and the sum. This will give me a tool to watch the number of connections and compare to the occurrences of problems. Naturally, since it is late on Friday afternoon, things are quieting down. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] T30File parameter in Universe 10.3.7
Yes, we ran into this limit some time ago at a client site, we set it to 1200. No problems. On Fri, 06 Jul 2012 14:15:00 -0400, Allen Egerton wrote: The default value for T30Files is 200, and it's described as controlling the number of dynamic files that can be open on a system at any point in time. Has anyone had practical experience with increasing this limit? And yes I'm asking because we just crashed into this wall --- and it wasn't pretty. Thank you. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
That sounds like the correct way to determine total license count. In
UV at least, there is also a more official way to do it via a built-in
subroutine called !GET.USER.COUNTS. I don't know if it exists in UD. I
have a simple program I can run at TCL that calls it and reports the
numbers:
0001: CALL !GET.USER.COUNTS (uv.users, max.uv.users, os.users)
0002: PRINT "Max UniVerse users: ":max.uv.users
0003: PRINT "Current UniVerse users: ":uv.users
0004: PRINT "OS users : ":os.users
-John
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of
charles_shaf...@ntn-bower.com
Sent: Friday, July 06, 2012 11:08 AM
To: U2 Users List
Subject: Re: [U2] Monitoring connections on Unidata
>> On UV, counting instances of uvapi_slave tells me how many unirpc
>> connections are active via UOJ. It doesn't include JDBC connections,
>> though. JDBC connections spawn a process called uvserver. If
capturing
>> every type of unirpc connection is important, you could determine the
>> PID of unirpcd and count its immediate children. I think this should
>> work:
>> PID=`ps -el | grep unirpcd | awk '{print $4}'` COUNT=`ps -el | grep
>> $PID | awk '{print $5}' | grep $PID | wc -l` echo $COUNT
>> -John
Thanks. That works great.
The problem I am having is that our MRP application does a lot of RPC
calls to our Unidata server. We have been having a lot of random
problems and I suspect that we are running out of connections. I am
looking for a way to show that for sure. I am recommending beefing up
our licenses and need to show some justification for the financial
expense.
Does the sum of the number of users plus the number of RPC connections
give me the number of licenses in use at the moment?
It seems that way. Right now there are 56 users listed in listuser.
There are 31 RPC connections. This is a total of 87. We have 96
licenses. And right now everybody seems to be happy. I expect that
when problems start occurring, I will check and find that the total of
the list user count and RPC connections will be greater than 96.
Does that sound right?
Charles Shaffer
Senior Analyst
NTN-Bower Corporation
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
You forgot the need to defragment, since someone suggested that my idea of using the intrinsic look-ahead ability is hampered by hard fragmentation. -Original Message- From: Rick Nuckolls To: 'U2 Users List' Sent: Fri, Jul 6, 2012 11:20 am Subject: Re: [U2] RESIZE - dynamic files Logically, the graphed solution to varying the split.load value with an -axis=modulus, y-axis=time_to_select_&_read_the_whole_file is going to be arabolic, having very slow performance at modulus=1 and modulus = # of records. If you actually want to find the precise low point, ignore all this bs, create a unch of files with copies of the same data, but different moduli, restart your ystem (including all disk drives & raid devices) in order to purge all buffers, nd then run the same program against each file. I think that we would all be urious about the results. Easier yet, just ignore the bs and use the defaults. :) -Rick -Original Message- rom: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] n Behalf Of Chris Austin ent: Friday, July 06, 2012 9:56 AM o: u2-users@listserver.u2ug.org ubject: Re: [U2] RESIZE - dynamic files o is there a performance increase in BASIC SELECTS by reducing overflow? Some eople are saying to reduce disk space to speed up the BASIC SELECT hile others say to reduce overflow.. I'm a bit confused. All of our programs hat read that table use a BASIC SELECT WITH.. for a BASIC select do you gain anything by reducing overflow? Chris To: u2-users@listserver.u2ug.org From: wjhon...@aol.com Date: Thu, 5 Jul 2012 20:12:21 -0400 Subject: Re: [U2] RESIZE - dynamic files A BASIC SELECT cannot use criteria at all. It is going to walk through every record in the file, in order. And that's the sticky wicket. That whole "in order" business. The disk drive controller has no clue on linked frames, but it *will* do ptimistic look aheads for you. So you are much better off, for BASIC SELECTs having nothing in overflow, at ll. :) That way, when you go to ask for the *next* frame, it will always be ontiguous, and already sitting in memory. -Original Message- From: Rick Nuckolls To: 'U2 Users List' Sent: Thu, Jul 5, 2012 4:43 pm Subject: Re: [U2] RESIZE - dynamic files Most disks and disk systems cache huge amounts of information these days, and, > epending on 20 factors or so, one solution will be better than another for a iven file. For the wholesale, SELECT F WITH, The fewest disk records will almost lways in. For files that have ~10 records/group and have ~10% of the groups verflowed, then perhaps 1% of record reads will do a second read for the verflow buffer because the target key was not in the primary group. Writing a > ew record would possibly hit the 10% mark for reading overflow buffers. But owering the split.load will increase the number of splits slightly, and ncrease the total number of groups considerably. What you have shown is that ou need to increase the the modulus (and select time) of a large file more han 0% in order to decrease the read and update times for you records 0.5% of the ime (assuming, that you have only reduced the number of overflow groups by 50%.) As Charles suggests, this is an interesting exercise, but your actual results ill rapidly change if you actually add /remove records from your file, change he load or number of files on your system, put in a new drive, cpu, memory oard, or install a new release of Universe, move to raid, etc. -Rick -Original Message- rom: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Wjhonson ent: Thursday, July 05, 2012 2:38 PM o: u2-users@listserver.u2ug.org ubject: Re: [U2] RESIZE - dynamic files he hardward "look ahead" of the disk drive reader will grab consecutive frames" into memory, since it assumes you'll want the "next" frame next. o the less overflow you have, the faster a full file scan will become. t least that's my theory ;) Original Message- rom: Rick Nuckolls o: 'U2 Users List' ent: Thu, Jul 5, 2012 2:29 pm ubject: Re: [U2] RESIZE - dynamic files hris, or the type of use that you described earlier; BASIC selects and reads, ducing overflow will have negligible performance benefit, especially compared changing the GROUP.SIZE back to 1 (2048) bytes. If you purge the file in latively small percentages, then it will never merge anyway (because you will ed to delete 20-30% of the file for that to happen with the mergeload at 50%, your optimum minimum modulus solution will probably be "how ever large it ows" The overhead for a group split is not as bad as it sounds unless your dates/sec count is extremely high, such as during a copy. f you do regular SELECT and SCANS of the entire file, then your goal should be > reduce the total disk size of the file, and not worry much about co
[U2] T30File parameter in Universe 10.3.7
The default value for T30Files is 200, and it's described as controlling the number of dynamic files that can be open on a system at any point in time. Has anyone had practical experience with increasing this limit? And yes I'm asking because we just crashed into this wall --- and it wasn't pretty. Thank you. -- Allen Egerton aeger...@pobox.com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
Chris, 10 years ago, when I was administering a UniVerse system, the answer would have been "minimize both to the best of your ability". But I don't know how UniVerse has changed in the interim, during which time I have been working on UniData systems, which are enormously different in their handling of records in groups from any other Pick-type system I have ever worked on (all of which were much more similar to UniVerse at that time). And when last I administered a UniVerse system, there were no dynamic files.. With that caveat, here are the factors: 1) a record in a UniVerse file that is stored in overflow is going to take 2 or more disk reads to retrieve if you are retrieving it by id. However, in a Basic select (structured as in Will's example, with no quotes, no "WITH" criteria), the system will walk through the file group by group, and will read each record, so yes, it will take 2 (or more, depending on how deeply that group is in overflow) reads to get the data, but it will have done the first read anyway to read those records - so for the Basic SELECT, you probably want to minimize the number of groups read to the extent that you can do so without putting many of the groups into overflow. 2) to add records to the file, you have to access the file by the record id, which means hashing the id to the group, then walking through the group to see if the id is already in use, and if not, adding the record to the end of the data area in use. So for that, you absolutely want to minimize the amount of overflow, because overflow slows you down on the 'adds'. 3) any sort/select or query read of the database will be slowed down significantly by overflow, but you said you don't do much of that anyway. Susan M. Lynch F. W. Davison & Company, Inc. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin Sent: 07/06/2012 12:56 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] RESIZE - dynamic files So is there a performance increase in BASIC SELECTS by reducing overflow? Some people are saying to reduce disk space to speed up the BASIC SELECT while others say to reduce overflow.. I'm a bit confused. All of our programs that read that table use a BASIC SELECT WITH.. for a BASIC select do you gain anything by reducing overflow? Chris > To: u2-users@listserver.u2ug.org > From: wjhon...@aol.com > Date: Thu, 5 Jul 2012 20:12:21 -0400 > Subject: Re: [U2] RESIZE - dynamic files > > > A BASIC SELECT cannot use criteria at all. > It is going to walk through every record in the file, in order. > And that's the sticky wicket. That whole "in order" business. > The disk drive controller has no clue on linked frames, but it *will* do optimistic look aheads for you. > So you are much better off, for BASIC SELECTs having nothing in overflow, at all. :) > That way, when you go to ask for the *next* frame, it will always be contiguous, and already sitting in memory. > > > > > > > > > -Original Message- > From: Rick Nuckolls > To: 'U2 Users List' > Sent: Thu, Jul 5, 2012 4:43 pm > Subject: Re: [U2] RESIZE - dynamic files > > > Most disks and disk systems cache huge amounts of information these days, and, > epending on 20 factors or so, one solution will be better than another for a > iven file. > For the wholesale, SELECT F WITH, The fewest disk records will almost always > in. For files that have ~10 records/group and have ~10% of the groups > verflowed, then perhaps 1% of record reads will do a second read for the > verflow buffer because the target key was not in the primary group. Writing a > ew record would possibly hit the 10% mark for reading overflow buffers. But > owering the split.load will increase the number of splits slightly, and > ncrease the total number of groups considerably. What you have shown is that > ou need to increase the the modulus (and select time) of a large file more than > 0% in order to decrease the read and update times for you records 0.5% of the > ime (assuming, that you have only reduced the number of overflow groups by > 50%.) > As Charles suggests, this is an interesting exercise, but your actual results > ill rapidly change if you actually add /remove records from your file, change > he load or number of files on your system, put in a new drive, cpu, memory > oard, or install a new release of Universe, move to raid, etc. > -Rick > -Original Message- > rom: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Wjhonson > ent: Thursday, July 05, 2012 2:38 PM > o: u2-users@listserver.u2ug.org > ubject: Re: [U2] RESIZE - dynamic files > > he hardward "look ahead" of the disk drive reader will grab consecutive > frames" into memory, since it assumes you'll want the "next" frame next. > o the less overflow you have, the faster a full file scan will become. > t least that's my theory ;) > > > Original Message- > rom: Rick Nu
Re: [U2] Monitoring connections on Unidata
>> On UV, counting instances of uvapi_slave tells me how many unirpc
>> connections are active via UOJ. It doesn't include JDBC connections,
>> though. JDBC connections spawn a process called uvserver. If
capturing
>> every type of unirpc connection is important, you could determine the
>> PID of unirpcd and count its immediate children. I think this should
>> work:
>> PID=`ps -el | grep unirpcd | awk '{print $4}'`
>> COUNT=`ps -el | grep $PID | awk '{print $5}' | grep $PID | wc -l`
>> echo $COUNT
>> -John
Thanks. That works great.
The problem I am having is that our MRP application does a lot of RPC
calls to our Unidata server. We have been having a lot of random problems
and I suspect that we are running out of connections. I am looking for a
way to show that for sure. I am recommending beefing up our licenses and
need to show some justification for the financial expense.
Does the sum of the number of users plus the number of RPC connections
give me the number of licenses in use at the moment?
It seems that way. Right now there are 56 users listed in listuser. There
are 31 RPC connections. This is a total of 87. We have 96 licenses. And
right now everybody seems to be happy. I expect that when problems start
occurring, I will check and find that the total of the list user count and
RPC connections will be greater than 96.
Does that sound right?
Charles Shaffer
Senior Analyst
NTN-Bower Corporation
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
Logically, the graphed solution to varying the split.load value with an x-axis=modulus, y-axis=time_to_select_&_read_the_whole_file is going to be parabolic, having very slow performance at modulus=1 and modulus = # of records. If you actually want to find the precise low point, ignore all this bs, create a bunch of files with copies of the same data, but different moduli, restart your system (including all disk drives & raid devices) in order to purge all buffers, and then run the same program against each file. I think that we would all be curious about the results. Easier yet, just ignore the bs and use the defaults. :) -Rick -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin Sent: Friday, July 06, 2012 9:56 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] RESIZE - dynamic files So is there a performance increase in BASIC SELECTS by reducing overflow? Some people are saying to reduce disk space to speed up the BASIC SELECT while others say to reduce overflow.. I'm a bit confused. All of our programs that read that table use a BASIC SELECT WITH.. for a BASIC select do you gain anything by reducing overflow? Chris > To: u2-users@listserver.u2ug.org > From: wjhon...@aol.com > Date: Thu, 5 Jul 2012 20:12:21 -0400 > Subject: Re: [U2] RESIZE - dynamic files > > > A BASIC SELECT cannot use criteria at all. > It is going to walk through every record in the file, in order. > And that's the sticky wicket. That whole "in order" business. > The disk drive controller has no clue on linked frames, but it *will* do > optimistic look aheads for you. > So you are much better off, for BASIC SELECTs having nothing in overflow, at > all. :) > That way, when you go to ask for the *next* frame, it will always be > contiguous, and already sitting in memory. > > > > > > > > > -Original Message- > From: Rick Nuckolls > To: 'U2 Users List' > Sent: Thu, Jul 5, 2012 4:43 pm > Subject: Re: [U2] RESIZE - dynamic files > > > Most disks and disk systems cache huge amounts of information these days, > and, > epending on 20 factors or so, one solution will be better than another for a > iven file. > For the wholesale, SELECT F WITH, The fewest disk records will almost > always > in. For files that have ~10 records/group and have ~10% of the groups > verflowed, then perhaps 1% of record reads will do a second read for the > verflow buffer because the target key was not in the primary group. Writing > a > ew record would possibly hit the 10% mark for reading overflow buffers. But > owering the split.load will increase the number of splits slightly, and > ncrease the total number of groups considerably. What you have shown is that > ou need to increase the the modulus (and select time) of a large file more > than > 0% in order to decrease the read and update times for you records 0.5% of the > ime (assuming, that you have only reduced the number of overflow groups by > 50%.) > As Charles suggests, this is an interesting exercise, but your actual results > ill rapidly change if you actually add /remove records from your file, change > he load or number of files on your system, put in a new drive, cpu, memory > oard, or install a new release of Universe, move to raid, etc. > -Rick > -Original Message- > rom: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Wjhonson > ent: Thursday, July 05, 2012 2:38 PM > o: u2-users@listserver.u2ug.org > ubject: Re: [U2] RESIZE - dynamic files > > he hardward "look ahead" of the disk drive reader will grab consecutive > frames" into memory, since it assumes you'll want the "next" frame next. > o the less overflow you have, the faster a full file scan will become. > t least that's my theory ;) > > > Original Message- > rom: Rick Nuckolls > o: 'U2 Users List' > ent: Thu, Jul 5, 2012 2:29 pm > ubject: Re: [U2] RESIZE - dynamic files > > hris, > or the type of use that you described earlier; BASIC selects and reads, > ducing overflow will have negligible performance benefit, especially compared > changing the GROUP.SIZE back to 1 (2048) bytes. If you purge the file in > latively small percentages, then it will never merge anyway (because you will > ed to delete 20-30% of the file for that to happen with the mergeload at 50%, > your optimum minimum modulus solution will probably be "how ever large it > ows" The overhead for a group split is not as bad as it sounds unless your > dates/sec count is extremely high, such as during a copy. > f you do regular SELECT and SCANS of the entire file, then your goal should > be > reduce the total disk size of the file, and not worry much about common > erflow. The important thing is that the file is dynamic, so you will never > counter the issues that undersized statically hashed files develop. > e have thousands of dyna
Re: [U2] Monitoring connections on Unidata
On UV, counting instances of uvapi_slave tells me how many unirpc
connections are active via UOJ. It doesn't include JDBC connections,
though. JDBC connections spawn a process called uvserver. If capturing
every type of unirpc connection is important, you could determine the
PID of unirpcd and count its immediate children. I think this should
work:
PID=`ps -el | grep unirpcd | awk '{print $4}'`
COUNT=`ps -el | grep $PID | awk '{print $5}' | grep $PID | wc -l`
echo $COUNT
-John
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of
charles_shaf...@ntn-bower.com
Sent: Friday, July 06, 2012 9:21 AM
To: U2 Users List
Subject: Re: [U2] Monitoring connections on Unidata
When I run # ps-A | grep 'udapi_slave' | wc -l
Am I getting a count of client connections?
Charles Shaffer
Senior Analyst
NTN-Bower Corporation
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
What do you mean a BASIC SELECT WITH... If you mean you are EXECUTE "SELECT CUSTOMER WITH..." that is not a BASIC SELECT whose syntax is only OPEN "CUSTOMER" TO F.CUSTOMER SELECT F.CUSTOMER no WITH -Original Message- From: Chris Austin To: u2-users Sent: Fri, Jul 6, 2012 10:23 am Subject: Re: [U2] RESIZE - dynamic files o is there a performance increase in BASIC SELECTS by reducing overflow? Some eople are saying to reduce disk space to speed up the BASIC SELECT hile others say to reduce overflow.. I'm a bit confused. All of our programs hat read that table use a BASIC SELECT WITH.. for a BASIC select do you gain anything by reducing overflow? Chris To: u2-users@listserver.u2ug.org From: wjhon...@aol.com Date: Thu, 5 Jul 2012 20:12:21 -0400 Subject: Re: [U2] RESIZE - dynamic files A BASIC SELECT cannot use criteria at all. It is going to walk through every record in the file, in order. And that's the sticky wicket. That whole "in order" business. The disk drive controller has no clue on linked frames, but it *will* do ptimistic look aheads for you. So you are much better off, for BASIC SELECTs having nothing in overflow, at ll. :) That way, when you go to ask for the *next* frame, it will always be ontiguous, and already sitting in memory. -Original Message- From: Rick Nuckolls To: 'U2 Users List' Sent: Thu, Jul 5, 2012 4:43 pm Subject: Re: [U2] RESIZE - dynamic files Most disks and disk systems cache huge amounts of information these days, and, > epending on 20 factors or so, one solution will be better than another for a iven file. For the wholesale, SELECT F WITH, The fewest disk records will almost lways in. For files that have ~10 records/group and have ~10% of the groups verflowed, then perhaps 1% of record reads will do a second read for the verflow buffer because the target key was not in the primary group. Writing a > ew record would possibly hit the 10% mark for reading overflow buffers. But owering the split.load will increase the number of splits slightly, and ncrease the total number of groups considerably. What you have shown is that ou need to increase the the modulus (and select time) of a large file more han 0% in order to decrease the read and update times for you records 0.5% of the ime (assuming, that you have only reduced the number of overflow groups by 50%.) As Charles suggests, this is an interesting exercise, but your actual results ill rapidly change if you actually add /remove records from your file, change he load or number of files on your system, put in a new drive, cpu, memory oard, or install a new release of Universe, move to raid, etc. -Rick -Original Message- rom: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Wjhonson ent: Thursday, July 05, 2012 2:38 PM o: u2-users@listserver.u2ug.org ubject: Re: [U2] RESIZE - dynamic files he hardward "look ahead" of the disk drive reader will grab consecutive frames" into memory, since it assumes you'll want the "next" frame next. o the less overflow you have, the faster a full file scan will become. t least that's my theory ;) Original Message- rom: Rick Nuckolls o: 'U2 Users List' ent: Thu, Jul 5, 2012 2:29 pm ubject: Re: [U2] RESIZE - dynamic files hris, or the type of use that you described earlier; BASIC selects and reads, ducing overflow will have negligible performance benefit, especially compared changing the GROUP.SIZE back to 1 (2048) bytes. If you purge the file in latively small percentages, then it will never merge anyway (because you will ed to delete 20-30% of the file for that to happen with the mergeload at 50%, your optimum minimum modulus solution will probably be "how ever large it ows" The overhead for a group split is not as bad as it sounds unless your dates/sec count is extremely high, such as during a copy. f you do regular SELECT and SCANS of the entire file, then your goal should be > reduce the total disk size of the file, and not worry much about common erflow. The important thing is that the file is dynamic, so you will never counter the issues that undersized statically hashed files develop. e have thousands of dynamically hashed files on our (Solaris) systems, with an > tremely low problem rate. ick Original Message- om: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Chris Austin nt: Thursday, July 05, 2012 11:21 AM : u2-users@listserver.u2ug.org bject: Re: [U2] RESIZE - dynamic files ick, ou are correct, I should be using the smaller size (I just haven't changed it t). Based on the reading I have done you should ly use the larger group size when the average record size is greater than 1000 > tes. s far as being better off with the defaults that's basically what I'm trying o est (as
Re: [U2] RESIZE - dynamic files
So is there a performance increase in BASIC SELECTS by reducing overflow? Some people are saying to reduce disk space to speed up the BASIC SELECT while others say to reduce overflow.. I'm a bit confused. All of our programs that read that table use a BASIC SELECT WITH.. for a BASIC select do you gain anything by reducing overflow? Chris > To: u2-users@listserver.u2ug.org > From: wjhon...@aol.com > Date: Thu, 5 Jul 2012 20:12:21 -0400 > Subject: Re: [U2] RESIZE - dynamic files > > > A BASIC SELECT cannot use criteria at all. > It is going to walk through every record in the file, in order. > And that's the sticky wicket. That whole "in order" business. > The disk drive controller has no clue on linked frames, but it *will* do > optimistic look aheads for you. > So you are much better off, for BASIC SELECTs having nothing in overflow, at > all. :) > That way, when you go to ask for the *next* frame, it will always be > contiguous, and already sitting in memory. > > > > > > > > > -Original Message- > From: Rick Nuckolls > To: 'U2 Users List' > Sent: Thu, Jul 5, 2012 4:43 pm > Subject: Re: [U2] RESIZE - dynamic files > > > Most disks and disk systems cache huge amounts of information these days, > and, > epending on 20 factors or so, one solution will be better than another for a > iven file. > For the wholesale, SELECT F WITH, The fewest disk records will almost > always > in. For files that have ~10 records/group and have ~10% of the groups > verflowed, then perhaps 1% of record reads will do a second read for the > verflow buffer because the target key was not in the primary group. Writing > a > ew record would possibly hit the 10% mark for reading overflow buffers. But > owering the split.load will increase the number of splits slightly, and > ncrease the total number of groups considerably. What you have shown is that > ou need to increase the the modulus (and select time) of a large file more > than > 0% in order to decrease the read and update times for you records 0.5% of the > ime (assuming, that you have only reduced the number of overflow groups by > 50%.) > As Charles suggests, this is an interesting exercise, but your actual results > ill rapidly change if you actually add /remove records from your file, change > he load or number of files on your system, put in a new drive, cpu, memory > oard, or install a new release of Universe, move to raid, etc. > -Rick > -Original Message- > rom: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Wjhonson > ent: Thursday, July 05, 2012 2:38 PM > o: u2-users@listserver.u2ug.org > ubject: Re: [U2] RESIZE - dynamic files > > he hardward "look ahead" of the disk drive reader will grab consecutive > frames" into memory, since it assumes you'll want the "next" frame next. > o the less overflow you have, the faster a full file scan will become. > t least that's my theory ;) > > > Original Message- > rom: Rick Nuckolls > o: 'U2 Users List' > ent: Thu, Jul 5, 2012 2:29 pm > ubject: Re: [U2] RESIZE - dynamic files > > hris, > or the type of use that you described earlier; BASIC selects and reads, > ducing overflow will have negligible performance benefit, especially compared > changing the GROUP.SIZE back to 1 (2048) bytes. If you purge the file in > latively small percentages, then it will never merge anyway (because you will > ed to delete 20-30% of the file for that to happen with the mergeload at 50%, > your optimum minimum modulus solution will probably be "how ever large it > ows" The overhead for a group split is not as bad as it sounds unless your > dates/sec count is extremely high, such as during a copy. > f you do regular SELECT and SCANS of the entire file, then your goal should > be > reduce the total disk size of the file, and not worry much about common > erflow. The important thing is that the file is dynamic, so you will never > counter the issues that undersized statically hashed files develop. > e have thousands of dynamically hashed files on our (Solaris) systems, with > an > tremely low problem rate. > ick > Original Message- > om: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] > n Behalf Of Chris Austin > nt: Thursday, July 05, 2012 11:21 AM > : u2-users@listserver.u2ug.org > bject: Re: [U2] RESIZE - dynamic files > ick, > ou are correct, I should be using the smaller size (I just haven't changed it > t). Based on the reading I have done you should > ly use the larger group size when the average record size is greater than > 1000 > tes. > s far as being better off with the defaults that's basically what I'm trying > to > est (as well as learn how linear hashing works). I was able > reduce my overflow by 18% and I only increased my empty groups by a very > all amount as well as only increased my file size > 8%. This in theory should be better f
Re: [U2] Monitoring connections on Unidata
When I run # ps-A | grep 'udapi_slave' | wc -l Am I getting a count of client connections? Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
What about connections coming from client applications? UniCall, UniObjects, etc. Do they use up licenses also? Charles Shaffer Senior Analyst NTN-Bower Corporation From: "Symeon Breen" To: "'U2 Users List'" , Date: 07/06/2012 10:18 AM Subject:Re: [U2] Monitoring connections on Unidata Sent by:u2-users-boun...@listserver.u2ug.org listuser ? -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: 06 July 2012 14:54 To: U2 Users List Subject: [U2] Monitoring connections on Unidata Is there a way to monitor the number of RPC connections being used at any given time on Unidata 7.1/Redhat 3? We are not using device licensing. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.2195 / Virus Database: 2437/5112 - Release Date: 07/05/12 ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Monitoring connections on Unidata
listuser ? -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of charles_shaf...@ntn-bower.com Sent: 06 July 2012 14:54 To: U2 Users List Subject: [U2] Monitoring connections on Unidata Is there a way to monitor the number of RPC connections being used at any given time on Unidata 7.1/Redhat 3? We are not using device licensing. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.2195 / Virus Database: 2437/5112 - Release Date: 07/05/12 ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Monitoring connections on Unidata
Is there a way to monitor the number of RPC connections being used at any given time on Unidata 7.1/Redhat 3? We are not using device licensing. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
On 05/07/12 23:58, Rick Nuckolls wrote: > Oops, I would of thought that if a file had, say 100,000 bytes, @ 70 percent > full, there would be 30,000 bytes "empty" or dead. Are you suggesting the > there would be 70,000 bytes of data and 42,000 bytes of dead space? Do you mean 100,000 bytes of disk space, or 100,000 bytes of data? I guess you are thinking that the file occupies, on disk, 100K. In which case it will have 70K of data and 30K of empty space. But if you are thinking that the file contains 100K of data, it will actually occupy 142K of disk space. So this particular option "wastes" 30% of the disk space it uses, but uses 42% extra space to what it optimally needed assuming perfect packing. I know, it's fun to try get your head round it :-) Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] RESIZE - dynamic files
The best thing I can say about the MINIMUM.MODULUS is that if you set it close to what you expect the file to need (at least for a while), when you start populating it from scratch, you will not lose the performance as the file keeps splitting. This can make an amazing difference in how long it take to initially populate the file. John -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson Sent: Thursday, July 05, 2012 5:41 PM To: U2 Users List Subject: Re: [U2] RESIZE - dynamic files Chris, I can appreciate what you are doing as an academic exercise. You seem happy how it looks at this moment, where, because you set "MINIMUM.MODULUS 118681", you ended up with a current load of 63%. But think about it: as you add records, the load will reach 70%, per "SPLIT.LOAD 70", then splits will keep occuring and current modlus with grow past 118681. MINIMUM.MODULUS will never matter again. (This was described as an ever-growing file.) If the current config is what you want, why not just set "SPLIT.LOAD 63 MINIMUM.MODULUS 1". That way the ratio that you like today will stay like this forever. MINIMUM.MODULUS will not matter unless data is deleted. It says to not shrink the file structure below that minimally allocated disk space, even if there is no data to occupy it. That's really all MINIMUM.MODULUS is for. Play with it all you want, because it puts you in a good place when some crisis happens. At the end of the day, with this file, you'll find your tuning won't matter much. Not a lot of help, but not much harm if you tweak it wrong, either. On 7/5/2012 1:20 PM, Chris Austin wrote: > Rick, > > You are correct, I should be using the smaller size (I just haven't > changed it yet). Based on the reading I have done you should only use the > larger group size when the average record size is greater than 1000 bytes. > > As far as being better off with the defaults that's basically what I'm > trying to test (as well as learn how linear hashing works). I was able > to reduce my overflow by 18% and I only increased my empty groups by a very > small amount as well as only increased my file size by 8%. This in theory > should be better for reads/writes than what I had before. > > To test the performance I need to write a ton of records and then capture the > output and compare the output using timestamps. > > Chris ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
