On 07/19/2012 07:22 AM, Brown, Larry - RD, St. Louis, MO wrote:
Hello, we have a job that was previously using Innovation's IAM file access
method. The file is over 6 million records. The job runs twice yearly and
usually takes an hour or less to complete. The product was removed to save on
SW costs, and the file was converted to VSAM. The programmer did not make any
other changes, and the job now takes over 10 hours to complete. I know the IAM
product is supposed to improve performance, but can't imagine it making the
difference between 1 and 14 hours run time. I'm suspecting there may have been
some JCL changes to blksize, buffers, and things like that required, but the
programmer is unaware of any of those changes he should have made. The job is
only reading the file. Does anybody have any ideas on where to start looking
for other changes that should have been made after converting from IAM to VSAM?
The programmer is reviewing his source code. Our performance support has not
suggested anything. Innovation claims %50-%80 re
duction i
n processing time, so maybe it is just a matter of IAM vs VSAM.(?)
Here are the JCL and VSAM definitions:
//MISC DD DSN=ASLP00.FT.MISC,DISP=SHR,
// AMP='BUFSP=409600,BUFND=181,BUFNI=9'
Cluster: 'ASLP00.FT.MISC' +
multi-volume
Data: 'ASLP00.FT.MISC.DATA' Data Volume:
PEST06 +
Index: 'ASLP00.FT.MISC.INDEX' Index Volume:
PEST06 +
Data Component Information: Current Allocation Options:
Device type: 3390 Load option:
SPEED
Organization: KSDS EXT-ADDR Write
check: NO
KSDS key length: 27 Buffer space:
10752
KSDS key location: 1 Erase on delete:
NO
Average record size: 80 Imbedded index:
NO
Maximum record size: 4084 Replicated index:
NO
Allocated Space: Unit Primary Secondary Reuse option:
YES
Data: CYLINDERS 2000 2000 Share
option: 2-3
Index: CYLINDERS 300 30 Spanned
records: NO
Dataset Date Information: Key ranges present:
NO
CLUSTER - ASLP00.FT.MISC Storage:
PESTBU
Data:
X4GB
Management:
MGTPSF
Owner ID:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Current Allocations in Tracks: Current Utilization in Tracks:
Allocated space: 240000 Used data space: 198827
( 83 %)
Allocated extents: 2 Used extents:
2 ( 100 %)
Allocation type: UNIQUE Prime records:
6,770,095
KSDS Index Allocation in Tracks: Deleted records:
3134
Allocated space: 4500 Inserted records:
8700
Number of records: 14795 Updated records:
706460
- - - - - - - - - - - Current Reorganization Information - - - - - - - - - - -
Data - Control Area Information: Control Interval Information:
Physical record size: 4096 Size-data: 4096
Index: 2560
Thanks,
Larry Brown
I'm not quite sure how BUFSP interacts with BUFND and BUFNI - I only
remember determining many years ago that it didn't make sense to use it
together with BUFND and BUFNI (which indirectly determine buffer space),
and that BUFND and BUFNI was always more useful than BUFSP.
I suspect with the almost 15,000 index CI's in this file that 9 index
buffers is way too low if you are doing random processing. The file
attributes you give don't seem to include Number of Index Levels, but
I'm sure with this many index records the number of levels is either 3
or 4 and the number of index buffers is probably too low to even include
all the higher level records, much less a useful working set of the
sequence set index CI's. Terribly inadequate VSAM buffering can easily
result in a factor of 100 increase in real time and CPU time
requirements for program execution. If random reads are involved, I
would be inclined to drastically increase the number of index buffers,
say to at least several hundred as a start.
If you are doing purely sequential processing, BUFND should probably be
large enough for two CA's, which would be 180*2 in this case. But there
are some cases where larger BUFND can actually degrade performance --
specifically, skip sequential processing (random positioning followed by
sequential reads) where only a few data CI's are actually processed. A
large BUFND is such a case can actually degrade performance because a
whole CA-worth of Data CI's may be read into buffers, only a few used,
and the rest discarded at the next random positioning. If you are doing
purely random reads, but have reason to believe there may be some
tendency to return to the same areas of the file, and the
skip-sequential case above is not an issue, you might also want to
experiment with adding hundreds to BUFND as well. LISTCAT statistics
before and after a run can show you whether the BUFNI/BUFND tuning is
having a significant effect.
Another possibility to consider is to read up on Batch LSR (BLSR) and
try specifying BLSR in JCL for the VSAM dataset. For random access,
BLSR is able to very efficiently manage thousands of buffers (uses
hashing rather than serial search to locate things), and when sufficient
real storage is available this can really speed up things for random
access. If the access is really straight sequential, then BLSR is not
so good, because it does not read-ahead to get all data CI's in a CA
with one I/O. On many systems these days and with programs able to
utilize data above the line, 65M of physical memory is not that big of a
deal. You could specify 15,000 BLSR index buffers in this case and be
guaranteed you would never read an index CI more than once, although
diminishing returns will probably be evident at fewer buffers. How many
data buffers might make sense depends totally on whether there is any
discernible locality of reference in the record access patterns, but
again BLSR can manage 1000's of buffers if it makes sense to do so.
Obviously, if you are instead in an environment that is real storage
constrained, you may have to be more circumspect in how many buffers you
specify as adding 100 MiB to your program's working set might have an
adverse impact on system paging, your throughput and that of others.
--
Joel C. Ewing, Bentonville, AR [email protected]
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
