Re: VSAM help wanted for random reads
Hi, This is the place to mention VSAM System Managed Buffering that could, possibly, auto-tune the access to BLSR if the open intention was set correctly by the programmer and enough region was available. Best Regards, Yifat -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Joel C. Ewing Sent: Wednesday, April 04, 2012 9:13 PM To: IBM-MAIN@bama.ua.edu Subject: Re: VSAM help wanted for random reads On 04/04/2012 10:37 AM, Chip Grantham wrote: We have an application like this, that is multiple record types in the same KSDS. We found that if we had a FD for the type '4' records and a FD for the type '5' records (that is two DDs pointing to the same file), that each kept a separate sequence set in storage and it ran faster. You might try it. Chip Grantham | Ameritas | Sr. IT Consultant | cgrant...@ameritas.com 5900 O Street, Lincoln NE 68510 | p: 402-467-7382 | c: 402-429-3579 | f: 402-325-4030 ... Unless you have something at your installation that automatically tunes VSAM buffer allocation, some kind of manual tuning in the JCL is almost always recommended, as the default VSAM buffer allocations tend to be terrible for performance. Just specifying an BUFFNI INDEX buffer count large enough to accommodate all index levels, plus additional buffers if the access pattern has multiple localities of reference, can do wonders for random access performance, even without going to BLSR. The default used to guarantee that random access to any VSAM file with data in more than one CA (and hence at least two levels of index) would require re-reading CI's for all the various index levels for each data record access. Just providing a few additional index buffers in such cases might be enough to cut the physical I/O's by a significant factor. -- Joel C. Ewing,Bentonville, AR jcew...@acm.org -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Peter, I've had this conversation in a few sites, and I've actually never come across a KSDS opened and updated with NSR (the usual COBOL strategy) that maintains integrity across a crash. This is a problem for KSDS with or without LSR and Deferred Writes and your backup recommendation is a requirement whether you use deferred writes or not. Remember the last time you recovered a KSDS that part way through a CA-split when the system pranged? VSAM NSR makes no attempt maintain Index and Data integrity to protect against a system or program crash. DEFERW does not change this exposure. Ron. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Farley, Peter x23353 Sent: Wednesday, April 04, 2012 6:01 AM To: IBM-MAIN@bama.ua.edu Subject: Re: [IBM-MAIN] VSAM help wanted for random reads Be very cautious using DEFERW with BLSR unless you already have a good backup of the KSDS immediately before the program starts. Recovery after a crash (program or system) will require a restore of the file, since DEFERW with a crash can leave the KSDS corrupted and sometimes unreadable, in part or in whole. BTDTGTTS. But it does speed up the update process substantially. If you decide to use it, add the measurements of both the time to take a backup just before starting the program and the program update time to compare to the non-DEFERW time. If you can allocate sufficient memory for buffers that will hold ALL of the CI's in the file, or at least all of the CI's that need an update (not always practical, of course), all of the updating will occur at CLOSE. Also note that I added a comma before the DEFERW in Ron's example. HTH Peter -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ron Hawkins Sent: Wednesday, April 04, 2012 12:35 AM To: IBM-MAIN@bama.ua.edu Subject: Re: VSAM help wanted for random reads Frank, It is terrific that you are getting an improvement with BLSR. I suspect you are using a vanilla copy of an example in the BLSR manual similar to Peter Farley's example in his post. The problem is these parms do not get the best performance from BLSR. The missing value is DEFERW. This is explained at http://publibz.boulder.ibm.com/cgi- bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?SH ELF=iea2bkb3 http://publibz.boulder.ibm.com/cgi- bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?S HELF=iea2bkb3DT=19940301112926CASE DT=19940301112926CASE= about three quarters of the way down the page. Using Peter's example I would suggest you should use: //MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL', // 'MSG=I','BUFND=256','BUFNI=10',DEFERW) //MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR If you do omit DEFERW a CI will be written to the KSDS every time you update it. It will stay buffered for a read buffer hit, but if you update 100 records in the CI you will write that CI 100 times. Not a huge problem for most modern DASD, but if you are running synchronous remote copy it will be painful. With DEFERW an updated CI will written when the LSR algorithm decides it is no longer active enough to remain in the LSR pool. You'll also see an unusual effect where all changed CI are written to KSDS at end of job. DEFERW also helps the performance of sequential inserts that do not use SIS, and CI and CA splits. It's an often omitted must-have for BLSR. NB it's a good practice to set BUFNI to the number of records in the Index component of the KSDS, plus 10%. Again based on the example, if you have a three level index with 11 index set records your sequence set will pollute the buffer hits on the low level index set records (the high level index set record is probably the most touched CI in the KSDS). -- This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Last time I looked, AIX with upgrade (required for the OP's described conditions) was a *VERY POOR* performer. However, with the small proportion of updates later described, this does not seem like it would have a disproportionate impact. snip Create an AIX pointing at the file with the AIX key being the account number followed by the record type. Change the update program's logic to access and update the AIX not the BASE File. All other programs access the BASE file as at present. Note that this requires you to look for ACCOUNT-NUMBER suffixed by 4 and look for equal-to or greater (ie: Accept the type 5 if it gets returned due to no type 4). Also you need to check the returned account number in case there is no 4 or 5 for the account number. /snip -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
+1 for DEFERW. But understand it defers the writes :-) ...
Once took a job down from 2M EXCPs to 0.5M with BLSR and then on down to
850 by adding DEFERW. 500 Control Intervals or so in the VSAM data set all
the I/O was to.
Cheers, Martin
Martin Packer,
Mainframe Performance Consultant, zChampion
Worldwide Banking Center of Excellence, IBM
+44-7802-245-584
email: martin_pac...@uk.ibm.com
Twitter / Facebook IDs: MartinPacker
Blog:
https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker
From:
Ron Hawkins ronjhawk...@sbcglobal.net
To:
IBM-MAIN@bama.ua.edu,
Date:
04/04/2012 05:34
Subject:
Re: VSAM help wanted for random reads
Sent by:
IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu
Frank,
It is terrific that you are getting an improvement with BLSR.
I suspect you are using a vanilla copy of an example in the BLSR manual
similar to Peter Farley's example in his post. The problem is these parms
do
not get the best performance from BLSR.
The missing value is DEFERW. This is explained at
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?SH
ELF=iea2bkb3
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?S
HELF=iea2bkb3DT=19940301112926CASE DT=19940301112926CASE= about three
quarters of the way down the page. Using Peter's example I would suggest
you
should use:
//MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL',
// 'MSG=I','BUFND=256','BUFNI=10' DEFERW)
//MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR
If you do omit DEFERW a CI will be written to the KSDS every time you
update
it. It will stay buffered for a read buffer hit, but if you update 100
records in the CI you will write that CI 100 times. Not a huge problem for
most modern DASD, but if you are running synchronous remote copy it will
be
painful. With DEFERW an updated CI will written when the LSR algorithm
decides it is no longer active enough to remain in the LSR pool. You'll
also
see an unusual effect where all changed CI are written to KSDS at end of
job.
DEFERW also helps the performance of sequential inserts that do not use
SIS,
and CI and CA splits. It's an often omitted must-have for BLSR.
NB it's a good practice to set BUFNI to the number of records in the Index
component of the KSDS, plus 10%. Again based on the example, if you have a
three level index with 11 index set records your sequence set will pollute
the buffer hits on the low level index set records (the high level index
set
record is probably the most touched CI in the KSDS).
Ron
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On
Behalf Of Frank Swarbrick
Sent: Tuesday, April 03, 2012 4:33 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: [IBM-MAIN] VSAM help wanted for random reads
Several good ideas given.
My sysprog installed BLSR and I got very good results:
Without BLSR:
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 1123K
1121K
.83 .11
23.2 1599506 TSTBAT
0
IEF404I
ICM06F - ENDED -
TIME=16.43.46 -ICM06F ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .83 TOTAL ELAPSED TIME=
23.2
With BLSR:
CSR020I
BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0,
SHRPOOL=14.
DDNAME=ICMMSTR
CSR022I
STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I
ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 10704
3532
.04 .00
1.1 73844
TSTBAT 0
0 0 0
IEF404I
ICM06F - ENDED -
TIME=16.55.07 -ICM06F ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.1
23.2 minutes versus 1.1 minutes. 1,123,000 EXCP versus 10,704 EXCP.
BTW, to Steve Comstock, there is no AIX on this file. Just a 17 byte
key
where the first byte is the record type ('0' - '5') and the remaining
16
is the
'actual' key in the format appropriate for that particular record type.
The idea of putting the record type at the end rather than the beginning
is an
interesting idea. Unless there's some way of doing that without having
to
change any programs I don't think we'd want to take the time. However I
am
interested enough to try it with this one program and see what effect it
has.
Thanks!
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions, send
email
to
mailto:lists...@bama.ua.edu lists...@bama.ua.edu with the message:
INFO
IBM-MAIN
Re: VSAM help wanted for random reads
Be very cautious using DEFERW with BLSR unless you already have a good backup of the KSDS immediately before the program starts. Recovery after a crash (program or system) will require a restore of the file, since DEFERW with a crash can leave the KSDS corrupted and sometimes unreadable, in part or in whole. BTDTGTTS. But it does speed up the update process substantially. If you decide to use it, add the measurements of both the time to take a backup just before starting the program and the program update time to compare to the non-DEFERW time. If you can allocate sufficient memory for buffers that will hold ALL of the CI's in the file, or at least all of the CI's that need an update (not always practical, of course), all of the updating will occur at CLOSE. Also note that I added a comma before the DEFERW in Ron's example. HTH Peter -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ron Hawkins Sent: Wednesday, April 04, 2012 12:35 AM To: IBM-MAIN@bama.ua.edu Subject: Re: VSAM help wanted for random reads Frank, It is terrific that you are getting an improvement with BLSR. I suspect you are using a vanilla copy of an example in the BLSR manual similar to Peter Farley's example in his post. The problem is these parms do not get the best performance from BLSR. The missing value is DEFERW. This is explained at http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?SH ELF=iea2bkb3 http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?S HELF=iea2bkb3DT=19940301112926CASE DT=19940301112926CASE= about three quarters of the way down the page. Using Peter's example I would suggest you should use: //MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL', // 'MSG=I','BUFND=256','BUFNI=10',DEFERW) //MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR If you do omit DEFERW a CI will be written to the KSDS every time you update it. It will stay buffered for a read buffer hit, but if you update 100 records in the CI you will write that CI 100 times. Not a huge problem for most modern DASD, but if you are running synchronous remote copy it will be painful. With DEFERW an updated CI will written when the LSR algorithm decides it is no longer active enough to remain in the LSR pool. You'll also see an unusual effect where all changed CI are written to KSDS at end of job. DEFERW also helps the performance of sequential inserts that do not use SIS, and CI and CA splits. It's an often omitted must-have for BLSR. NB it's a good practice to set BUFNI to the number of records in the Index component of the KSDS, plus 10%. Again based on the example, if you have a three level index with 11 index set records your sequence set will pollute the buffer hits on the low level index set records (the high level index set record is probably the most touched CI in the KSDS). -- This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
It looked good when I sent it!
Let me try this...
Before BLSR:
- -TIMINGS
(MINS.)-- -PAGING COUNTS
-STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK
SERV WORKLOAD PAGE SWAP VIO SWAPS
-PROC01 STEP01 00 1123K 1121K .83 .11 23.2
1599506 TSTBAT 0 0 0 0
IEF404I ICM06F - ENDED -
TIME=16.43.46
-ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .83 TOTAL
ELAPSED TIME= 23.2
With BLSR:
CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0,
SHRPOOL=14. DDNAME=ICMMSTR
CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
+BMC2700 API BYPASSED -- NO PLUSIN DD PRESENT
BMC
- -TIMINGS
(MINS.)-- -PAGING COUNTS
-STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK
SERV WORKLOAD PAGE SWAP VIO SWAPS
-PROC01 STEP01 00 10704 3532 .04 .00 1.1
73844 TSTBAT 0 0 0 0
IEF404I ICM06F - ENDED -
TIME=16.55.07
-ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .04 TOTAL
ELAPSED TIME= 1.1
If that doesn't format correctly them I am at a loss.
Frank
From: Ed Gould edgould1...@comcast.net
To: IBM-MAIN@bama.ua.edu
Sent: Tuesday, April 3, 2012 6:24 PM
Subject: Re: VSAM help wanted for random reads
Frank,
Thanks for reporting back.
One request is there somehow to paste the results so they match up with the
headers ?
Ed
On Apr 3, 2012, at 6:33 PM, Frank Swarbrick wrote:
Several good ideas given.
My sysprog installed BLSR and I got very good results:
Without BLSR:
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEP RC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 1123K
1121K
.83 .11
23.2 1599506 TSTBAT
0
IEF404I
ICM06F - ENDED -
TIME=16.43.46
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .83 TOTAL ELAPSED TIME=
23.2
With BLSR:
CSR020I
BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14.
DDNAME=ICMMSTR
CSR022I
STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I
ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEP RC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 10704
3532
.04 .00
1.1 73844
TSTBAT 0
0 0 0
IEF404I
ICM06F - ENDED -
TIME=16.55.07
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .04 TOTAL ELAPSED
TIME= 1.1
23.2 minutes versus 1.1 minutes. 1,123,000 EXCP versus 10,704 EXCP.
BTW, to Steve Comstock, there is no AIX on this file. Just a 17 byte key
where the first byte is the record type ('0' - '5') and the remaining 16
is the 'actual' key in the format appropriate for that particular record
type.
The idea of putting the record type at the end rather than the beginning is
an interesting idea. Unless there's some way of doing that without having
to change any programs I don't think we'd want to take the time. However I
am interested enough to try it with this one program and see what effect it
has.
Thanks!
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
We have an application like this, that is multiple record types in the same KSDS. We found that if we had a FD for the type '4' records and a FD for the type '5' records (that is two DDs pointing to the same file), that each kept a separate sequence set in storage and it ran faster. You might try it. Chip Grantham | Ameritas | Sr. IT Consultant | cgrant...@ameritas.com 5900 O Street, Lincoln NE 68510 | p: 402-467-7382 | c: 402-429-3579 | f: 402-325-4030 *** This message may contain confidential information intended only for the use of the addressee(s) named above and may contain information that is legally privileged. If you are not the addressee, or the person responsible for delivering it to the addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message immediately thereafter. Thank you. *** -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
That was our original thought, but since we have to open it as I-O it wouldn't work for us. Well, not without shareoptions 4. Thanks! Frank - Original Message - From: Chip Grantham cgrant...@ameritas.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, April 4, 2012 9:37 AM Subject: Re: VSAM help wanted for random reads We have an application like this, that is multiple record types in the same KSDS. We found that if we had a FD for the type '4' records and a FD for the type '5' records (that is two DDs pointing to the same file), that each kept a separate sequence set in storage and it ran faster. You might try it. Chip Grantham | Ameritas | Sr. IT Consultant | cgrant...@ameritas.com 5900 O Street, Lincoln NE 68510 | p: 402-467-7382 | c: 402-429-3579 | f: 402-325-4030 *** This message may contain confidential information intended only for the use of the addressee(s) named above and may contain information that is legally privileged. If you are not the addressee, or the person responsible for delivering it to the addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message immediately thereafter. Thank you. *** -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Tested with DEFERW=YES (that is the proper syntax, by the way. Commas are optional.) It didn't gain me much, as I am only updating 32 of 61459 records, so I'm not going to worry about it at this time. With BLSR DEFERW=YES: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10649 3485 .04 .00 1.2 73938 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.36.39 -ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS466 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95EC792E74FAC04' REC-RETRIEVED--10103 FREESPC-19886080 With BLSR DEFERW=NO: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10661 4459 .04 .00 1.2 74300 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.53.06 -ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS471 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95ECB3FB8134004' REC-RETRIEVED--10103 FREESPC-19886080 I did a delete/define/repro from source for the ICMMSTR file before each run. Thanks for everyone's help! Frank - Original Message - From: Farley, Peter x23353 peter.far...@broadridge.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, April 4, 2012 7:00 AM Subject: Re: VSAM help wanted for random reads Be very cautious using DEFERW with BLSR unless you already have a good backup of the KSDS immediately before the program starts. Recovery after a crash (program or system) will require a restore of the file, since DEFERW with a crash can leave the KSDS corrupted and sometimes unreadable, in part or in whole. BTDTGTTS. But it does speed up the update process substantially. If you decide to use it, add the measurements of both the time to take a backup just before starting the program and the program update time to compare to the non-DEFERW time. If you can allocate sufficient memory for buffers that will hold ALL of the CI's in the file, or at least all of the CI's that need an update (not always practical, of course), all of the updating will occur at CLOSE. Also note that I added a comma before the DEFERW in Ron's example. HTH Peter -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ron Hawkins Sent: Wednesday, April 04, 2012 12:35 AM To: IBM-MAIN@bama.ua.edu Subject: Re: VSAM help wanted for random reads Frank, It is terrific that you are getting an improvement with BLSR. I suspect you are using a vanilla copy of an example in the BLSR manual similar to Peter Farley's example in his post. The problem is these parms do not get the best performance from BLSR. The missing value is DEFERW. This is explained at http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?SH ELF
Re: VSAM help wanted for random reads
On 04/04/2012 10:37 AM, Chip Grantham wrote: We have an application like this, that is multiple record types in the same KSDS. We found that if we had a FD for the type '4' records and a FD for the type '5' records (that is two DDs pointing to the same file), that each kept a separate sequence set in storage and it ran faster. You might try it. Chip Grantham | Ameritas | Sr. IT Consultant | cgrant...@ameritas.com 5900 O Street, Lincoln NE 68510 | p: 402-467-7382 | c: 402-429-3579 | f: 402-325-4030 ... Unless you have something at your installation that automatically tunes VSAM buffer allocation, some kind of manual tuning in the JCL is almost always recommended, as the default VSAM buffer allocations tend to be terrible for performance. Just specifying an BUFFNI INDEX buffer count large enough to accommodate all index levels, plus additional buffers if the access pattern has multiple localities of reference, can do wonders for random access performance, even without going to BLSR. The default used to guarantee that random access to any VSAM file with data in more than one CA (and hence at least two levels of index) would require re-reading CI's for all the various index levels for each data record access. Just providing a few additional index buffers in such cases might be enough to cut the physical I/O's by a significant factor. -- Joel C. Ewing,Bentonville, AR jcew...@acm.org -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Not to suggest YOU should've done this but you can get useful statistics out of SMF 62 and 64 that would've told you how much write activity you had - and a lot else besides. Cheers, Martin Martin Packer, Mainframe Performance Consultant, zChampion Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Frank Swarbrick frank.swarbr...@yahoo.com To: IBM-MAIN@bama.ua.edu, Date: 04/04/2012 16:59 Subject: Re: VSAM help wanted for random reads Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Tested with DEFERW=YES (that is the proper syntax, by the way. Commas are optional.) It didn't gain me much, as I am only updating 32 of 61459 records, so I'm not going to worry about it at this time. With BLSR DEFERW=YES: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEPRC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10649 3485 .04 .001.2 73938 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.36.39 -ICM06F ENDED. NAME-ICM06TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS466 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95EC792E74FAC04' REC-RETRIEVED--10103 FREESPC-19886080 With BLSR DEFERW=NO: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEPRC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10661 4459 .04 .001.2 74300 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.53.06 -ICM06F ENDED. NAME-ICM06TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS471 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95ECB3FB8134004' REC-RETRIEVED--10103 FREESPC-19886080 I did a delete/define/repro from source for the ICMMSTR file before each run. Thanks for everyone's help! Frank - Original Message - From: Farley, Peter x23353 peter.far...@broadridge.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, April 4, 2012 7:00 AM Subject: Re: VSAM help wanted for random reads Be very cautious using DEFERW with BLSR unless you already have a good backup of the KSDS immediately before the program starts. Recovery after a crash (program or system) will require a restore of the file, since DEFERW with a crash can leave the KSDS corrupted and sometimes unreadable, in part or in whole. BTDTGTTS. But it does speed up the update process substantially. If you decide to use it, add the measurements of both the time to take a backup just before starting the program and the program update time to compare to the non-DEFERW time. If you can allocate sufficient memory for buffers that will hold ALL of the CI's in the file, or at least all of the CI's that need an update (not always practical, of course), all of the updating will occur at CLOSE. Also note that I added a comma before the DEFERW in Ron's
Re: VSAM help wanted for random reads
I possibly could if I had the slightest idea how to do that. Is that something an applications developer might have access to, or only a sysprog? Thanks, Frank - Original Message - From: Martin Packer martin_pac...@uk.ibm.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, April 4, 2012 1:34 PM Subject: Re: VSAM help wanted for random reads Not to suggest YOU should've done this but you can get useful statistics out of SMF 62 and 64 that would've told you how much write activity you had - and a lot else besides. Cheers, Martin Martin Packer, Mainframe Performance Consultant, zChampion Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Frank Swarbrick frank.swarbr...@yahoo.com To: IBM-MAIN@bama.ua.edu, Date: 04/04/2012 16:59 Subject: Re: VSAM help wanted for random reads Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Tested with DEFERW=YES (that is the proper syntax, by the way. Commas are optional.) It didn't gain me much, as I am only updating 32 of 61459 records, so I'm not going to worry about it at this time. With BLSR DEFERW=YES: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10649 3485 .04 .00 1.2 73938 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.36.39 -ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS466 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95EC792E74FAC04' REC-RETRIEVED--10103 FREESPC-19886080 With BLSR DEFERW=NO: CSR020I BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14. DDNAME=ICMMSTR CSR022I STRNO=16, ACB RMODE31=ALL, RMODE31=ALL. DDNAME=ICMMSTR +CSR021I ACB CONVERTED TO USE VSAM LSR. DDNAME=ICMMSTR - -TIMINGS (MINS.)-- -PAGING COUNTS -STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV WORKLOAD PAGE SWAP VIO SWAPS -PROC01 STEP01 00 10661 4459 .04 .00 1.2 74300 TSTBAT 0 0 0 0 IEF404I ICM06F - ENDED - TIME=09.53.06 -ICM06F ENDED. NAME-ICM06 TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.2 STATISTICS REC-TOTAL--61459 SPLITS-CI--0 EXCPS471 REC-DELETED0 SPLITS-CA--0 EXTENTS1 REC-INSERTED---0 FREESPACE-%CI-20 SYSTEM-TIMESTAMP: REC-UPDATED---32 FREESPACE-%CA-10 X'C95ECB3FB8134004' REC-RETRIEVED--10103 FREESPC-19886080 I did a delete/define/repro from source for the ICMMSTR file before each run. Thanks for everyone's help! Frank - Original Message - From: Farley, Peter x23353 peter.far...@broadridge.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, April 4, 2012 7:00 AM Subject: Re: VSAM help wanted for random reads Be very cautious using DEFERW with BLSR unless you already have a good backup of the KSDS immediately before the program starts. Recovery after a crash (program or system) will require a restore of the file, since DEFERW with a crash can leave the KSDS corrupted and sometimes unreadable, in part or in whole. BTDTGTTS. But it does speed up the update process
Re: VSAM help wanted for random reads
VSAM keeps up with this stuff too. Wouldn't hurt to do a reorg every once in blue moon too! In a message dated 4/4/2012 6:12:16 P.M. Central Daylight Time, martin_pac...@uk.ibm.com writes: out of SMF 62 and 64 that would've told you how much write activity you had - and a lot else besides -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
At 16:33 -0700 on 04/03/2012, Frank Swarbrick wrote about Re: VSAM help wanted for random reads: The idea of putting the record type at the end rather than the beginning is an interesting idea.Ý Unless there's some way of doing that without having to change any programs I don't think we'd want to take the time.Ý However I am interested enough to try it with this one program and see what effect it has. Create an AIX pointing at the file with the AIX key being the account number followed by the record type. Change the update program's logic to access and update the AIX not the BASE File. All other programs access the BASE file as at present. Note that this requires you to look for ACCOUNT-NUMBER suffixed by 4 and look for equal-to or greater (ie: Accept the type 5 if it gets returned due to no type 4). Also you need to check the returned account number in case there is no 4 or 5 for the account number. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4' records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May
2010) more than one sequence set can be in memory, but with MVS this appears to
not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Frank,
Try using batch local shared resources (SUBSYS=BLSR) for the file, and specify
at least twice as many index buffers for it as you have levels in the KSDS.
For example:
//MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL',
// 'MSG=I','BUFND=256','BUFNI=10')
//MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR
Where HLQ.MY.KSDS has 3 index levels. The BUFND counts data CI's that can be
resident at the same time, adjust to suit your needs. The BUFNI is for index
buffers (again CI's). Don't forget to increase REGION to account for the
additional buffers in memory.
Or you might try the new-fangled way, using SMB (system managed buffering). I
haven't yet found that to be more helpful to performance than BLSR, but it
might be worth your while to investigate it. See the JCL Reference manual for
the parameters to use.
HTH
Peter
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of
Frank Swarbrick
Sent: Tuesday, April 03, 2012 5:24 PM
To: IBM-MAIN@bama.ua.edu
Subject: VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4' records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May
2010) more than one sequence set can be in memory, but with MVS this appears to
not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
This message and any attachments are intended only for the use of the addressee
and may contain information that is privileged and confidential. If the reader
of the message is not the intended recipient or an authorized representative of
the intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Run the 4's then run the 5's. It can't be that simple.
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of
Frank Swarbrick
Sent: Tuesday, April 03, 2012 4:24 PM
To: IBM-MAIN@bama.ua.edu
Subject: VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4' records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May
2010) more than one sequence set can be in memory, but with MVS this appears to
not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
One (of many) possible explanations is that there are more hits on the type 5
and more rewrites. Depending on your VSAM options, a rewrite can be -very-
expensive.
Try a comparison run without the type 4 logic.
Also try BLSR and DISP=OLD. That makes the best use of VSAM buffering.
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of
Frank Swarbrick
Sent: Tuesday, April 03, 2012 4:24 PM
To: IBM-MAIN@bama.ua.edu
Subject: VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4' records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May
2010) more than one sequence set can be in memory, but with MVS this appears to
not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@bama.ua.edu with the message: INFO IBM-MAIN
NOTICE: This electronic mail message and any files transmitted with it are
intended
exclusively for the individual or entity to which it is addressed. The message,
together with any attachment, may contain confidential and/or privileged
information.
Any unauthorized review, use, printing, saving, copying, disclosure or
distribution
is strictly prohibited. If you have received this message in error, please
immediately advise the sender by reply email and delete all copies.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Interesting. I've wanted to use the SUBSYS parm, but never had cause.
However we must not have BLSR installed:
IEFC746I SUBSYSTEM BLSR DOES NOT EXIST
I'll look further into it.
Thanks,
Frank
From: Farley, Peter x23353 peter.far...@broadridge.com
To: IBM-MAIN@bama.ua.edu
Sent: Tuesday, April 3, 2012 3:33 PM
Subject: Re: VSAM help wanted for random reads
Frank,
Try using batch local shared resources (SUBSYS=BLSR) for the file, and specify
at least twice as many index buffers for it as you have levels in the KSDS.
For example:
//MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL',
// 'MSG=I','BUFND=256','BUFNI=10')
//MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR
Where HLQ.MY.KSDS has 3 index levels. The BUFND counts data CI's that can be
resident at the same time, adjust to suit your needs. The BUFNI is for index
buffers (again CI's). Don't forget to increase REGION to account for the
additional buffers in memory.
Or you might try the new-fangled way, using SMB (system managed buffering). I
haven't yet found that to be more helpful to performance than BLSR, but it
might be worth your while to investigate it. See the JCL Reference manual for
the parameters to use.
HTH
Peter
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of
Frank Swarbrick
Sent: Tuesday, April 03, 2012 5:24 PM
To: IBM-MAIN@bama.ua.edu
Subject: VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one
minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4'
records and the 'type 5' records it's constantly going back and forth, loading
the other sequence set from disk. He says on VSE (which we were up until
May 2010) more than one sequence set can be in memory, but with MVS this
appears to not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
This message and any attachments are intended only for the use of the
addressee and may contain information that is privileged and confidential. If
the reader of the message is not the intended recipient or an authorized
representative of the intended recipient, you are hereby notified that any
dissemination of this communication is strictly prohibited. If you have
received this communication in error, please notify us immediately by e-mail
and delete the message and any attachments from your system.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Should have mentioned, that is a thought we've considered. And we may end up
doing. Just wanted a better way, if there is one.
Frank
From: Fred Hoffman fhoff...@tad.org
To: IBM-MAIN@bama.ua.edu
Sent: Tuesday, April 3, 2012 3:34 PM
Subject: Re: VSAM help wanted for random reads
Run the 4's then run the 5's. It can't be that simple.
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of
Frank Swarbrick
Sent: Tuesday, April 03, 2012 4:24 PM
To: IBM-MAIN@bama.ua.edu
Subject: VSAM help wanted for random reads
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is
found then it tries again for the same account, but with record type '5'. If
either the '4' or '5' type is found it updates it and rewrites the record
(assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is removed,
so that it only ever looks for type '4' records, it takes only about one
minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4'
records and the 'type 5' records it's constantly going back and forth, loading
the other sequence set from disk. He says on VSE (which we were up until
May 2010) more than one sequence set can be in memory, but with MVS this
appears to not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
On 4/3/2012 3:23 PM, Frank Swarbrick wrote:
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of
arecord on the KSDS file. The key is, essentially, a concatenation of a record
type code ('4' or '5') and an account number. If no record of type '4' is found
then it tries again for the same account, but with record type '5'. If either
the '4' or '5' type is found it updates it and rewrites the record (assuming the
data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is
removed,so that it only ever looks for type '4' records, it takes only about
one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4' records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May 2010)
more than one sequence set can be in memory, but with MVS this appears to not be
the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
Well, I think we have a vocabulary problem. A VSAM KSDS only
has one primary index, and the lowest level is the sequence
set. You can't have two primary indexes, so you can only
have one sequence set. It's not clear how your key is constructed,
but you can get the whole sequence set in memory by using the
right JCL DD parameters.
First do a listc all against the cluster. Jot down these
values: CI/CA for data component (shows as PHYRECS/TRK),
and Levels in the index; then try this:
if this process is batch, and no CICS processing is going
on against the data set, consider adding:
BUFNDI=#Index_levels,BUFND=(2*PHYRECS/TRK)+1
If, on the other hand, your type '4' records are accessed as
the primary key and your type '5' are accessed as part of
an alternate index, you might need a different collection of
DD parameters. (Long shot guess: add BUFNDI on the DD statement
for your AIX).
--
Kind regards,
-Steve Comstock
The Trainer's Friend, Inc.
303-355-2752
http://www.trainersfriend.com
* To get a good Return on your Investment, first make an investment!
+ Training your people is an excellent investment
* Try our tool for calculating your Return On Investment
for training dollars at
http://www.trainersfriend.com/ROI/roi.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Have you looked at adding BUFNI
and/or BUFND to the JCL DD ?
Do you have any vendor products doing VSAM tuning ? possibly doing buffer
tuning..
Regards,
Doug
Sent from my iPhone
On Apr 3, 2012, at 17:55, Steve Comstock st...@trainersfriend.com wrote:
On 4/3/2012 3:23 PM, Frank Swarbrick wrote:
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of
arecord on the KSDS file. The key is, essentially, a concatenation of a
record
type code ('4' or '5') and an account number. If no record of type '4' is
found
then it tries again for the same account, but with record type '5'. If either
the '4' or '5' type is found it updates it and rewrites the record (assuming
the
data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is
removed,so that it only ever looks for type '4' records, it takes only about
one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4'
records
and the 'type 5' records it's constantly going back and forth, loading the
other sequence set from disk. He says on VSE (which we were up until May
2010)
more than one sequence set can be in memory, but with MVS this appears to not
be
the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
Well, I think we have a vocabulary problem. A VSAM KSDS only
has one primary index, and the lowest level is the sequence
set. You can't have two primary indexes, so you can only
have one sequence set. It's not clear how your key is constructed,
but you can get the whole sequence set in memory by using the
right JCL DD parameters.
First do a listc all against the cluster. Jot down these
values: CI/CA for data component (shows as PHYRECS/TRK),
and Levels in the index; then try this:
if this process is batch, and no CICS processing is going
on against the data set, consider adding:
BUFNDI=#Index_levels,BUFND=(2*PHYRECS/TRK)+1
If, on the other hand, your type '4' records are accessed as
the primary key and your type '5' are accessed as part of
an alternate index, you might need a different collection of
DD parameters. (Long shot guess: add BUFNDI on the DD statement
for your AIX).
--
Kind regards,
-Steve Comstock
The Trainer's Friend, Inc.
303-355-2752
http://www.trainersfriend.com
* To get a good Return on your Investment, first make an investment!
+ Training your people is an excellent investment
* Try our tool for calculating your Return On Investment
for training dollars at
http://www.trainersfriend.com/ROI/roi.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
For right now, apply to '4' accounts, save misses to a file, then
close and open for input and process the '5' account and report the
misses.
I assume the input is sorted into ascending order for the fast
results. How about changing the key definition so the account number
is in front and the account type (4 or 5) is at the end. Should be
able to change the copylib and recompile, group move would have to be
broken into individual moves. You can still print and display them
the other way.
On Tue, Apr 3, 2012 at 4:23 PM, Frank Swarbrick
frank.swarbr...@yahoo.com wrote:
Current program does the following (in COBOL!).
- Opens KSDS for I-O
- Sequentially processes another file (non-VSAM).
- For each record in the non-VSAM file it attempts a random (keyed) read of a
record on the KSDS file. The key is, essentially, a concatenation of a
record type code ('4' or '5') and an account number. If no record of type
'4' is found then it tries again for the same account, but with record type
'5'. If either the '4' or '5' type is found it updates it and rewrites the
record (assuming the data changed).
This takes about 16 minutes to run. However if the 'type 5' logic is
removed, so that it only ever looks for type '4' records, it takes only about
one minute.
Our resident VSAM expert says it has something to do with the VSAM sequence
set and the fact that only one is kept in memory at a time. Since the same
sequence set doesn't appear to be able to be used for both the 'type 4'
records and the 'type 5' records it's constantly going back and forth,
loading the other sequence set from disk. He says on VSE (which we were up
until May 2010) more than one sequence set can be in memory, but with MVS
this appears to not be the case.
Hopefully that reason is accurate. Is there a solution? We thought about
having two COBOL FDs pointing to two DDs, where both DDs pointed to the same
KSDS. This would probably(?) work, except for the fact of the update.
Thanks,
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Several good ideas given.
My sysprog installed BLSR and I got very good results:
Without BLSR:
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEP RC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 1123K
1121K
.83 .11
23.2 1599506 TSTBAT
0
IEF404I
ICM06F - ENDED -
TIME=16.43.46
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .83 TOTAL ELAPSED TIME=
23.2
With BLSR:
CSR020I
BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0, SHRPOOL=14.
DDNAME=ICMMSTR
CSR022I
STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I
ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEP RC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 10704
3532
.04 .00
1.1 73844
TSTBAT 0
0 0 0
IEF404I
ICM06F - ENDED -
TIME=16.55.07
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .04 TOTAL ELAPSED
TIME= 1.1
23.2 minutes versus 1.1 minutes. 1,123,000 EXCP versus 10,704 EXCP.
BTW, to Steve Comstock, there is no AIX on this file. Just a 17 byte key where
the first byte is the record type ('0' - '5') and the remaining 16 is the
'actual' key in the format appropriate for that particular record type.
The idea of putting the record type at the end rather than the beginning is an
interesting idea. Unless there's some way of doing that without having to
change any programs I don't think we'd want to take the time. However I am
interested enough to try it with this one program and see what effect it has.
Thanks!
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Frank,
Thanks for reporting back.
One request is there somehow to paste the results so they match up
with the headers ?
Ed
On Apr 3, 2012, at 6:33 PM, Frank Swarbrick wrote:
Several good ideas given.
My sysprog installed BLSR and I got very good results:
Without BLSR:
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 1123K
1121K
.83 .11
23.2 1599506 TSTBAT
0
IEF404I
ICM06F - ENDED -
TIME=16.43.46
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .83 TOTAL ELAPSED TIME=
23.2
With BLSR:
CSR020I
BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0,
SHRPOOL=14.
DDNAME=ICMMSTR
CSR022I
STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I
ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 10704
3532
.04 .00
1.1 73844
TSTBAT 0
0 0 0
IEF404I
ICM06F - ENDED -
TIME=16.55.07
-ICM06F
ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .04 TOTAL ELAPSED
TIME= 1.1
23.2 minutes versus 1.1 minutes. 1,123,000 EXCP versus 10,704 EXCP.
BTW, to Steve Comstock, there is no AIX on this file. Just a 17
byte key where the first byte is the record type ('0' - '5') and
the remaining 16 is the 'actual' key in the format appropriate for
that particular record type.
The idea of putting the record type at the end rather than the
beginning is an interesting idea. Unless there's some way of doing
that without having to change any programs I don't think we'd want
to take the time. However I am interested enough to try it with
this one program and see what effect it has.
Thanks!
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: VSAM help wanted for random reads
Frank,
It is terrific that you are getting an improvement with BLSR.
I suspect you are using a vanilla copy of an example in the BLSR manual
similar to Peter Farley's example in his post. The problem is these parms do
not get the best performance from BLSR.
The missing value is DEFERW. This is explained at
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?SH
ELF=iea2bkb3
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA5J600/3.1.1?S
HELF=iea2bkb3DT=19940301112926CASE DT=19940301112926CASE= about three
quarters of the way down the page. Using Peter's example I would suggest you
should use:
//MYKSDS DD SUBSYS=(BLSR,'DDNAME=MYKSDS#','RMODE31=ALL',
// 'MSG=I','BUFND=256','BUFNI=10' DEFERW)
//MYKSDS# DD DSN=HLQ.MY.KSDS,DISP=SHR
If you do omit DEFERW a CI will be written to the KSDS every time you update
it. It will stay buffered for a read buffer hit, but if you update 100
records in the CI you will write that CI 100 times. Not a huge problem for
most modern DASD, but if you are running synchronous remote copy it will be
painful. With DEFERW an updated CI will written when the LSR algorithm
decides it is no longer active enough to remain in the LSR pool. You'll also
see an unusual effect where all changed CI are written to KSDS at end of
job.
DEFERW also helps the performance of sequential inserts that do not use SIS,
and CI and CA splits. It's an often omitted must-have for BLSR.
NB it's a good practice to set BUFNI to the number of records in the Index
component of the KSDS, plus 10%. Again based on the example, if you have a
three level index with 11 index set records your sequence set will pollute
the buffer hits on the low level index set records (the high level index set
record is probably the most touched CI in the KSDS).
Ron
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On
Behalf Of Frank Swarbrick
Sent: Tuesday, April 03, 2012 4:33 PM
To: IBM-MAIN@bama.ua.edu
Subject: Re: [IBM-MAIN] VSAM help wanted for random reads
Several good ideas given.
My sysprog installed BLSR and I got very good results:
Without BLSR:
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 1123K
1121K
.83 .11
23.2 1599506 TSTBAT
0
IEF404I
ICM06F - ENDED -
TIME=16.43.46 -ICM06F ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .83 TOTAL ELAPSED TIME=
23.2
With BLSR:
CSR020I
BUFSI=1024, BUFSD=20480, BUFNI=10, BUFND=256, HBUFNI=0, HBUFND=0,
SHRPOOL=14.
DDNAME=ICMMSTR
CSR022I
STRNO=16, ACB RMODE31=ALL, RMODE31=ALL.
DDNAME=ICMMSTR
+CSR021I
ACB CONVERTED TO USE VSAM LSR.
DDNAME=ICMMSTR
-
-TIMINGS
(MINS.)--
-PAGING COUNTS
-STEPNAME
PROCSTEPRC EXCP
CONN
TCB SRB
CLOCK SERV
WORKLOAD PAGE SWAP VIO SWAPS
-PROC01
STEP01 00 10704
3532
.04 .00
1.1 73844
TSTBAT 0
0 0 0
IEF404I
ICM06F - ENDED -
TIME=16.55.07 -ICM06F ENDED.
NAME-ICM06
TOTAL TCB CPU TIME= .04 TOTAL ELAPSED TIME= 1.1
23.2 minutes versus 1.1 minutes. 1,123,000 EXCP versus 10,704 EXCP.
BTW, to Steve Comstock, there is no AIX on this file. Just a 17 byte key
where the first byte is the record type ('0' - '5') and the remaining 16
is the
'actual' key in the format appropriate for that particular record type.
The idea of putting the record type at the end rather than the beginning
is an
interesting idea. Unless there's some way of doing that without having to
change any programs I don't think we'd want to take the time. However I
am
interested enough to try it with this one program and see what effect it
has.
Thanks!
Frank
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to
mailto:lists...@bama.ua.edu lists...@bama.ua.edu with the message: INFO
IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
