Re: Confused on DFSORT SORTWK
I think my post was confusing. Normally you don't need DSNTYPE=LARGE to
use multiple extents. But in your specific case, you had primary and
secondary of 3335 cylinders (50025 tracks) so a second extent would put you
over the 65535 limit. So that's why I said you needed DSNTYPE=LARGE to use
multiple extents. Sorry for the confusion.
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email: bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 05/11/2012
03:13:32 PM:
From: Scott Barry sba...@sbbworks.com
To: IBM-MAIN@bama.ua.edu,
Date: 05/11/2012 03:14 PM
Subject: Re: Confused on DFSORT SORTWK
Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu
Why would I be forced to code DSNTYPE=LARGE, rather than using the
WRKSEC=Y parameter in SORTCNTL along with //SORTWKnn DDs with
secondary allocation specified? I am also curious about the SEC=nn
parameter, as well.
For my interests, I have chosen DSNTYPE=LARGE, however one still
must remember the 16 extent per volume limitation, so it's oh so
important to code a sufficiently large enough secondary amount (and/
or primary as well) so that a one-volume allocation can even exceed
the 65K track limitation and still be within 16 extents.
Scott Barry
SBBWorks, Inc.
On Tue, 1 May 2012 15:52:41 -0400, David Betten bet...@us.ibm.com
wrote:
Just to clarify, DFSORT can cetrainly use more than the 1st extent, it
just
can't use multiple volumes. As long as DSNTYPE=LARGE is coded on the DD
statement, each of the SORTWK DDs should be able to use multiple
extents.
Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to
get 3 extents of 3335 cylinders on each work data set if the volumes are
empty.
From: Neil Duffee nduf...@uottawa.ca
To: IBM-MAIN@bama.ua.edu,
Date: 05/01/2012 03:14 PM
Subject: Re: Confused on DFSORT SORTWK
Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu
In addition to all the other responses you got regarding DynAlloc, I
wanted to add the nudge that DFSORT only uses the **1st** extent in
SortWk DD statements. Thus, SPACE=(CYL,(3335,3335)) is the same as
SPACE=(CYL,3335) which, in your example, only got you 10*'3335
cylinders or the equivalent to 10 mod-3s. So you'd need to have 27
DD statements to use your entire pool.
signature = 6 lines follows
Neil Duffee, Joe Sysprog, uOttawa, Ottawa, Ont, Canada
telephone:1 613 562 5800 x4585 fax:1 613 562 5161
mailto:NDuffee of uOttawa.ca http:/ /aix1.uOttawa.ca/ ~nduffee
How *do* you plan for something like that? Guardian Bob, Reboot
For every action, there is an equal and opposite criticism.
Systems Programming: Guilty, until proven innocent John Norgauer
2004
-Original Message-
From: Mark Pace [mailto:pacemainl...@gmail.com]
Sent: April 27, 2012 08:43
Subject: Confused on DFSORT SORTWK
Trying to sort a fairly large RMF data file. 20,000 + cylinders.
I keep running out of SORTWK space. I have 9 3390-9 volumes in my
SMS temporary storage pool. But I just noticed that no matter how
many SORTWKxx DDs I add it only allocates file on the first 5 temp
storage volumes.
Anyone have an idea how I make DFSORT use more volumes?
//SORTWK01 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(3335,3335))
[snip]
ICE046A E SORT CAPACITY EXCEEDED - RECORD COUNT 2459859
[snip]
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email: bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Confused on DFSORT SORTWK
DFSORT should be allocating as large format. Do you have VIO=YES as your default. That might be preventing DFSORT from allocating as large format. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 05/06/2012 02:55:53 PM: From: Skip Robinson jo.skip.robin...@sce.com To: IBM-MAIN@bama.ua.edu, Date: 05/06/2012 02:57 PM Subject: Re: Confused on DFSORT SORTWK Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu (Reconstructing thread from archive) I read the posts in this thread but deleted them because they were not immediately pertinent to our shop. Then last night I got woke up with a similar problem. The default DYNALLOC setting of 3 was insufficient to complete a sort involving an unusually large input. We got ICE083A D RESOURCES WERE UNAVAILABLE FOR DYNAMIC ALLOCATION OF WORK DATA SETS (064K) - ICE753I FWK=(0,0) SWK=(0,0) TWK=(0,0) RWK=(0,0) TOTAL=(0,0) BLK=56600 ICE248I ATTEMPTED TO ALLOCATE 19389MB OF DISK WORK SPACE ON 3 WORK DATA SETS ICE249I SUCCESSFULLY ALLOCATED 0MB OF DISK WORK SPACE ON 0 WORK DATA SETS (Last line is probably the world's most disappointing 'success' message ever.) In researching the problem, I read that DFSORT for some time has been capable of using large or extended format data sets so that the 64K limit flagged in ICE083A does not apply. What is required to induce DFSORT to use modern work data sets for dynamic allocation? . . JO.Skip Robinson SCE Infrastructure Technology Services Electric Dragon Team Paddler SHARE MVS Program Co-Manager 626-302-7535 Office 323-715-0595 Mobile jo.skip.robin...@sce.com Just to clarify, DFSORT can cetrainly use more than the 1st extent, it just can't use multiple volumes. As long as DSNTYPE=LARGE is coded on the DD statement, each of the SORTWK DDs should be able to use multiple extents. Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to get 3 extents of 3335 cylinders on each work data set if the volumes are empty. From: Neil Duffee nduf...@uottawa.ca To: IBM-MAIN@bama.ua.edu, Date: 05/01/2012 03:14 PM Subject: Re: Confused on DFSORT SORTWK Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu In addition to all the other responses you got regarding DynAlloc, I wanted to add the nudge that DFSORT only uses the **1st** extent in SortWk DD statements. Thus, SPACE=(CYL,(3335,3335)) is the same as SPACE=(CYL,3335) which, in your example, only got you 10*'3335 cylinders or the equivalent to 10 mod-3s. So you'd need to have 27 DD statements to use your entire pool. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: SELECTING SMF RECORDS FOR 1 DAY
Try this in your SYSIN //SYSIN DD * INDD(INDD1,OPTIONS(DUMP)) OUTDD(OUTDD1,TYPE(14:15)) DATE(2012125,2012125) /* In my example, the job would select only records from today. Change the 125 to the day you need to select. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 05/04/2012 07:13:28 AM: From: John Dawes jhn_da...@yahoo.com.au To: IBM-MAIN@bama.ua.edu, Date: 05/04/2012 07:13 AM Subject: SELECTING SMF RECORDS FOR 1 DAY Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu G'Day, I am following up on a problem regarding the corruption of a dsn. I have the SMF records for the day, however since the daily tapes are merged into a weekly tape I am having a problem (space issues) trying to extract only the type 14 15 records for that particular date. I looked at - OSMVS System Management Facilities - the doc for an example but I didn't find any. Can someone suggest what I can try. Below is my job which I am trying to run. //STEP1 EXEC PGM=IFASMFDP //INDD1 DD DSN=SYS2.SMFDATA.WEEK.G1584V00,DISP=SHR //OUTDD1 DD DSN=SM11, // DISP=(,PASS),UNIT=SYSDA,SPACE=(CYL,(150,150),RLSE), // DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=4096) //SYSPRINT DD SYSOUT=* //SYSIN DD * INDD(INDD1,OPTIONS(DUMP)) OUTDD(OUTDD1,TYPE(14:15)) /* //STEP2 EXEC PGM=IDCAMS,TIME=1440 //DD1 DD DSN=SM11, // DISP=(OLD,DELETE), // DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=4096) //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSIN DD * PRINT INFILE(DD1) DUMP /* // -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Confused on DFSORT SORTWK
Just to clarify, DFSORT can cetrainly use more than the 1st extent, it just can't use multiple volumes. As long as DSNTYPE=LARGE is coded on the DD statement, each of the SORTWK DDs should be able to use multiple extents. Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to get 3 extents of 3335 cylinders on each work data set if the volumes are empty. From: Neil Duffee nduf...@uottawa.ca To: IBM-MAIN@bama.ua.edu, Date: 05/01/2012 03:14 PM Subject: Re: Confused on DFSORT SORTWK Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu In addition to all the other responses you got regarding DynAlloc, I wanted to add the nudge that DFSORT only uses the **1st** extent in SortWk DD statements. Thus, SPACE=(CYL,(3335,3335)) is the same as SPACE=(CYL,3335) which, in your example, only got you 10*'3335 cylinders or the equivalent to 10 mod-3s. So you'd need to have 27 DD statements to use your entire pool. signature = 6 lines follows Neil Duffee, Joe Sysprog, uOttawa, Ottawa, Ont, Canada telephone:1 613 562 5800 x4585 fax:1 613 562 5161 mailto:NDuffee of uOttawa.ca http:/ /aix1.uOttawa.ca/ ~nduffee How *do* you plan for something like that? Guardian Bob, Reboot For every action, there is an equal and opposite criticism. Systems Programming: Guilty, until proven innocent John Norgauer 2004 -Original Message- From: Mark Pace [mailto:pacemainl...@gmail.com] Sent: April 27, 2012 08:43 Subject: Confused on DFSORT SORTWK Trying to sort a fairly large RMF data file. 20,000 + cylinders. I keep running out of SORTWK space. I have 9 3390-9 volumes in my SMS temporary storage pool. But I just noticed that no matter how many SORTWKxx DDs I add it only allocates file on the first 5 temp storage volumes. Anyone have an idea how I make DFSORT use more volumes? //SORTWK01 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(3335,3335)) [snip] ICE046A E SORT CAPACITY EXCEEDED - RECORD COUNT 2459859 [snip] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Confused on DFSORT SORTWK
��z{S���}�ĝ� �xjǺ�*'���O*^��m��Z�w!j�
Re: Confused on DFSORT SORTWK
��z{S���}�ĝ� �xjǺ�*'���O*^��m��Z�w!j�
Test
Sorry about this post. Trying to determine why my posts are coming out garbled. Dave Betten -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Test
I think for some reason it's related to when I do a Reply with History. I recently went to a newer version of Lotus Notes and suspect something is now different. Sorry for the bother to everyone. David, The two messages of yours which came through to me garbled had this in the mail headers: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 The test message had this instead: Content-Type: text/plain; charset=US-ASCII HTH Peter Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Confused on DFSORT SORTWK
Just to clarify, DFSORT supports large format data sets for sortworks thus allowing them to use more than 64K tracks. DFSORT's dynamic allocation allocates them as large format. For JCL allocated sort works you need to specify DSNTYPE=LARGE on the DD statements and DFSORT will support that. My rule of thumb for SORTWK - space is: You need about 1 1/2 to 2 times the size of SORTIN for your SORTWKs (OK, somebody will probably correct me or point me to where there's documentation about the real values, but that's what I have used in the past). A SORTWK dataset's maximum size is (or used to be when I last messed around with that a few years ago) 64K tracks (4368 CYL). So, going out on my old limb, I guesstimate that for 20,000 CYL of input file you may need between 30,000 and 40,000 CYL of SORTWK space or at least 10 SORTWK DDs. More smaller ones might be better as they allow the disks to be filled more completely. If the disks are SMS managed, an overflow - pool might be a good backstop. Regards, Ulrich Krueger Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Confused on DFSORT SORTWK
��z{S���}�ĝ� �xjǺ�*'���O*^��m��Z�w!j�
Re: z/OS 1.11 DFSORT ICE046A
There can be many reasons they are now suddenly getting ICE046A. Can you send a few sysouts from jobs receiving the ICE046A to the DFSORT Hotline, dfs...@us.ibm.com ? Once we determine why they are getting the ICE046A, we can determine the best approach for addressing it in your environment. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 03/26/2012 12:49:24 PM: From: subscribe ibm-main Jerry Bergman bergm...@co.kern.ca.us To: IBM-MAIN@bama.ua.edu, Date: 03/26/2012 12:54 PM Subject: z/OS 1.11 DFSORT ICE046A Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Hi, We recently upgraded from 1.9 to 1.11. It went well, but now several of our production jobs are dying with ICE406A - Sort Capacity Exceeded. The JCL has 3 sortwork DDs with an allocation of CYL(1,1) or (2,1). Quite frankly I'm surprised they ever worked. When I tell the application folks they need to follow the recommendation from the doc for ICE046A, namely increase their sortwork size, their response is that I need to set some parameter so they don't have to change their JCL and if I ask someone will surely know what that parameter is and tell me. So I am asking. Thanks, Jerry Bergman County of Kern bergm...@co.kern.ca.us -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Writing article on telework/telecommuting
One thing I'll add to that is that if your internet service periodially drops, it's a real pain if you're connected to a host 3270 session. For example, my wife primarally does email and web browsing while working from home. So if our internet signal drops for a few minutes and then comes back, she's not likely to even notice. However, if I'm scrolling through code or a hex dump and the sevice drops for just a few seconds, it's a major headache getting loging back on and hoping my sesson reconnects to where I was. Our latest VPN client seems to offer a bit better recovery from that by maintaining the session but a few years ago it was a major headache for me. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 03/04/2012 08:49:57 AM: From: Martin Packer martin_pac...@uk.ibm.com To: IBM-MAIN@bama.ua.edu, Date: 03/04/2012 10:01 AM Subject: Re: Writing article on telework/telecommuting Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu One experience from teleworking which should appeal to mainframers: By and large 3270 is the least demanding data stream - so TSO / ISPF goes fast even on broadband as crummy as mine. (It's all the other junk that runs really slowly when the wet string dries out.) Now I may be in a minority but I bet this counts for lots of people. Anyhow, having telecommuted for more than 10 years I'm looking forward to this article: You are not alone is a useful thing to hear. :-) Cheers, Martin Martin Packer, Mainframe Performance Consultant, zChampion Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Gabe Goldberg g...@gabegold.com To: IBM-MAIN@bama.ua.edu, Date: 03/03/2012 21:43 Subject: Writing article on telework/telecommuting Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu I'm writing article for Destination z http://destinationz.org/ on telework/telecommuting. I think this partitions in two dimensions -- technology vs. mindset and worker vs. employer. There's abundant information -- and blather -- about this subject. But Destination z is mainframe focused so I'm especially interested in relevant System z tips for all four quadrants: technology/mindset/worker/employer. Again, this is a tips article so won't include positive/negative anecdotes. But they're still welcome -- they can suggest tips, they're interesting, and I might write a longer piece on this sometime. As usual, extra credit for sending to me directly (in addition to list, if you're so inclined) so I needn't pluck from digests. Thanks, as always.. -- Gabriel Goldberg, Computers and Publishing, Inc. g...@gabegold.com 3401 Silver Maple Place, Falls Church, VA 22042 (703) 204-0433 LinkedIn: http://www.linkedin.com/in/gabegoldTwitter: GabeG0 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal sort
I think you really need to figure out why your JCL sortwork is being directed to VIO. Have you tried using a large space allocation. Usually ACS routines set a MAXSIZE threshold for directing to a VIO storage group. If you can stop the SORT work data set from being directed to VIO, then DFSORT will not try to re-allocate it to SORTDK. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/22/2012 12:17:33 PM: [image removed] Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal sort Staffan Tylen to: IBM-MAIN 02/22/2012 12:19 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Sorry Otto, SORTWK01 in the JCL doesn't help. DFSORT then starts to allocate SORTDKnn files as real files, not VIO as is the case with dynamic allocation. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal sort
I can't give specific recommendations without knowing more about these sorts being executed and the environment. But perhaps I can offer a little info that will be useful. The reason the JCL SORTWKxx data sets get reallocated to SORTDKxx is because VIO=NO is in effect. This is the installation default we recommend. So when the work data sets are allocated to VIO, DFSORT attempts to re-allocate them to SORTDKxx data sets using a different unit name in hopes they will be directed to disk instead of VIO. Unfortunately, VIO cannot be specified as a run time option so you cannot change it for just this job. But perhaps you can code JCL SORTWKs using a unit that is not directed to VIO. I don't know how your ACS routines are set up but usually a VIO storage class is assigned based on the primary space allocation size. So perhaps if you allocated 1 or 2 SORTWK dds with a larger storage size, they'd be assigned to disk and thus DFSORT would not try to re-allocate them. When no JCL sortwks are present, then DFSORT's dynamic allocation is used. Usually, even if the sort is expected to complete in memory, work data sets are still allocated since we can't be entirely sure of the file size being passed. You could at least reduce the number of work data sets being passed by using an OPTION statement with DYNALLOC=(,2) which would reduce the number of work data sets allocated. There might be some other options we can come up with if we had more info. Perhaps you can send the entire joblog from an execution of this program to our DFSORT hot line, dfs...@us.ibm.com and we can take a look. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/20/2012 12:22:44 PM: [image removed] was: Abend S0C4 in an internal sort Staffan Tylen to: IBM-MAIN 02/20/2012 12:23 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List I wish to continue this thread as it's related. I'm now running my program with internal calls to DFSORT and it works fine except for one issue that I have, namely that I get a lot of job log messages generated as a result of dynamic allocation of VIO sortwork files. DFSORT is invoked 1000's of times by the program (yes, I know but there is nothing I can do about that) and each time it creates a bunch of job log messages for VIO allocations. For performance I want all sorting to be performed in memory, and I don't really care at the moment whether it's in hiperspace or dataspace or whatever. I successfully used the FILSZ=Ennn parameter until I ran out of space, so I increased the value to match the reality and since then I cannot get rid of the dynamic allocation messages. I've tried DYNALLOC=OFF, HIPRMAX=0, DSPSIZE=0, DSA=0, FILSZ=U0, you name it! in all sorts of combinations in an attempt to prevent dynamic allocation of work files, all in vain. I've tried region sizes far! too big including REGION=0M. I've also tried preallocating SORTWKxx to VIO data sets, but DFSORT then allocates its own SORTDKxx data sets instead :( So how can I successfully sort my records in storage without using dynamic allocation with all the rubbish job log messages as a result? Many thanks, Staffan -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Lots of *MASTER* CPU during a big sort
DFSORT was likely utilizing available central storage for memory object or Hiperspace sorting. Normally I have not seen that cause spikes in Master cpu utilization but perhaps there was a spike in paging that drove the Master cpu. If so, there are DFSORT installation defaults that can be set to limit DFSORT's use of central storage to minimize high paging. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/25/2012 10:08:43 AM: [image removed] Lots of *MASTER* CPU during a big sort David Andrews to: IBM-MAIN 01/25/2012 10:14 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List I glanced at the SDSF 'DA' frame this morning and noticed one of our applications programmers was sorting a large file - something like 400,000 tracks. The system was otherwise lightly loaded and SDSF was reporting 2M real storage frames in use by his job. But what also caught my eye was unusually large CPU consumption by the *MASTER* address space: 60-75% continuously while his sort was executing. He cancelled it, and *MASTER* went back to its usual docile self. What is a large DFSort with 2M+ real storage doing to consume that much time in *MASTER*? Is this a characteristic of memory objects? -- David Andrews A. Duda Sons, Inc. david.andr...@duda.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: DFSORT manual humour
Back when we updated the pubs for V1R10, I thought I got rid of all those outdated references. Looks like I missed one. I'll talk to Frank about updating that one for our next release. To your question, you are correct. If DFSORT's dynamic allocation is being used, then DFSORT calculates the total work space requirement and spreads it across the number of work data sets specified in the DYNALOC parameter. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/21/2012 10:30:43 PM: [image removed] DFSORT manual humour R.S. to: IBM-MAIN 01/21/2012 10:39 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Quote from DFSORT manual: --- For best performance, specify an emulated 3390-9 device (such as RAMAC) or another high-speed IBM disk device as the default, and avoid specifying a tape, virtual (VIO), or real 3390-9 devices as the default. --- I have to admit the manual is a little bit obsolete - it is dated on 2009. However I'm still under impression of high-speed RAMAC devices. BTW, now seriously: I can specify number of dynamically allocated work datasets via DYNALOC, but the size of those dataset is controlled by DFSORT and user cannot change it. Is it true? -- Radoslaw Skorupka Lodz, Poland -- Tre tej wiadomoci moe zawiera informacje prawnie chronione Banku przeznaczone wycznie do uytku subowego adresata. Odbiorc moe by jedynie jej adresat z wyczeniem dostpu osób trzecich. Jeeli nie jeste adresatem niniejszej wiadomoci lub pracownikiem upowanionym do jej przekazania adresatowi, informujemy, e jej rozpowszechnianie, kopiowanie, rozprowadzanie lub inne dziaanie o podobnym charakterze jest prawnie zabronione i moe by karalne. Jeeli otrzymae t wiadomo omykowo, prosimy niezwocznie zawiadomi nadawc wysyajc odpowied oraz trwale usun t wiadomo wczajc w to wszelkie jej kopie wydrukowane lub zapisanena dysku. This e-mail may contain legally privileged information of the Bank and is intended solely for business use of the addressee. This e- mail may only be received by the addressee and may not be disclosed to any third parties. If you are not the intended addressee of this e-mail or the employee authorised to forward it to the addressee, be advised that any dissemination, copying, distribution or any other similar activity is legally prohibited and may be punishable. If you received this e-mail by mistake please advise the sender immediately by using the reply facility in your e-mail software and delete permanently this e-mail including any copies of it either printed or saved to hard drive. BRE Bank SA, 00-950 Warszawa, ul. Senatorska 18, tel. +48 (22) 829 00 00, fax +48 (22) 829 00 33, www.brebank.pl, e-mail: i...@brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237, NIP: 526-021-50-88. Wedug stanu na dzie 01.01.2012 r. kapita zakadowy BRE Banku SA (w caoci wpacony) wynosi 168.410.984 zotych. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Number of SORT WORKs needed
If they are MOD-27s, you should be able to use more than 4300 cylinders on each volume. Just make sure to code DSNTYPE=LARGE on the JCL SORTWK statements. For a sort this large, I would also make sure your DFSORT installation default for DSA is set very large, like DSA=512. If you run into problems with this sort, feel free to send a note to our DFSORT hotline, dfs...@us.ibm.com Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/20/2012 10:15:53 AM: [image removed] Number of SORT WORKs needed af dc to: IBM-MAIN 01/20/2012 10:21 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hello, we have an DB2 unload that needs to be sorted, it has 900.000 CYLs, I'm detemining how many sortwkxx must I put on jcl to perform this...And according to my calculation using 255 sortwk files with 4300 cyls as primary allocation it will not be enough.Storage group for temporary files has a lot of free space with mod27 available. 900.000 * 2 = 1.800.000 cyls / 4300 = 419 sortwk !! Z/Os V1.12 Can you pls give me an help ?? Any hint is welcome. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: z/OS Tag Sort
Hello Yifet, Sorry for the delayed reply but I was on vacation. I really can't go into specifics of the algorithm since that is part of our proprietary internals. We generally recommend that our customers use MAINSIZE=MAX and let DFSORT's dynamic storage adjustment calculate the optimal amount of virtual storage. I can give you some general ranges of what DFSORT would calculate as optimal for various file sizes. File SizeOptimal virtual storage 32GB ~ 64MB 64GB ~100MB 128GB ~150MB 256GB ~200MB 512GB ~256MB 1TB~350MB These are just approximations of what you'd observe if you executed sorts of these sizes with no restrictions on region, MAINSIZE=MAX and DSA large enough not to limit. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 10/18/2011 07:39:10 AM: [image removed] Re: z/OS Tag Sort Yifat Oren to: IBM-MAIN 10/18/2011 07:50 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hi David, You wrote: .. That exit is limiting the sort to just 24MB of virtual storage when the optimum amount would be closer to 200MB. The data size was 360 GB (336m records). Can you please share the formula you've used to determine the optimum amount is around 200MB? The DFSORT Tuning Guide (1.12) seems to think 2GB is the upper limit when it comes to recommending minimum virtual storage settings :) Thanks, Yifat Oren -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: z/OS Tag Sort
John sent me his complete sysout. The Intermediate Merge is the main cause of the high elapsed time. However, in John's case, just increasing DSA is not going to help because there is an installation exit (ICEIEXIT) controlling the amount of virtual storage DFSORT can allocate. That exit is limiting the sort to just 24MB of virtual storage when the optimum amount would be closer to 200MB. As for the size of the sort, 336 million records (360GB), I have seen much larger sorts than that so I know DFSORT can handle it. We've actually executed sorts with record counts in the billions. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 10/14/2011 06:59:11 AM: [image removed] Re: z/OS Tag Sort Elardus Engelbrecht to: IBM-MAIN 10/14/2011 07:23 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List John Blythe Reid wrote: ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED Good Eye Catcher. Thanks for stating it here. ICE054I 0 RECORDS - IN: 336718372, OUT: 336718372 ICE134I 0 NUMBER OF BYTES SORTED: 378660163010 ICE253I 0 RECORDS SORTED - PROCESSED: 336718372, EXPECTED: 367987636 Oh, that's just a few records... :-D No seriously, that is a very serious lot of records. I really doubt that I even reached that quantity in ONE sort job in my whole career... ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 1124, EXPECTED: 1050 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 14030580 , TRACKS USED: 14022900 Are you using static or dynamic work datasets? In my humble opinion, I would go to static work datasets (SORTWKxx DD), but then this is my opinion and preference. Perhaps it is not suitable for you. ICE090I 0 OUTPUT LRECL = 2100, BLKSIZE = 32760, TYPE = VB (SDB) Perhaps you could try out this system determined blocksize of 27998? ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES Zeroes only? Where are your ICE093I and ICE156I messages? Can't you post them here too, please? I would like to add to Paolo Cacciari's reply (increase DSA): Could you perhaps increase your job REGION, say to 512M or so? Also consider adding these options, which helped me to sort out big sort jobs: OPTION DYNSPC=512,SIZE=E9,MAINSIZE=MAX The DFSORT Team may add better recommendations. There are no sort exits being used. There are no specific limitations on sorting by WLM or any other process. Good. I'm glad. Hopes that helps much to SORT out your problem!!! :-D Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: z/OS Tag Sort
ICE162I 0 ICEIEXIT CHANGED ONE OR MORE OPTIONS IN EFFECT Unfortunately we don't display what options were changed. Also in the ICE132I Options message I saw IEXIT=Y, TEXIT=Y which indicate both an installation and a termination exit are in use on this system. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 10/14/2011 08:35:35 AM: [image removed] Re: z/OS Tag Sort Elardus Engelbrecht to: IBM-MAIN 10/14/2011 08:37 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List David Betten bet...@us.ibm.com wrote: The Intermediate Merge is the main cause of the high elapsed time. However, in John's case, just increasing DSA is not going to help because there is an installation exit (ICEIEXIT) controlling the amount of virtual storage DFSORT can allocate. That exit is limiting the sort to just 24MB of virtual storage when the optimum amount would be closer to 200MB. Interesting. Are there any ICE message(s) showing that an exit is really in use/called? Just curious if you don't mind, please. As for the size of the sort, 336 million records (360GB), I have seen much larger sorts than that so I know DFSORT can handle it. We've actually executed sorts with record counts in the billions. Uhhh, I'm feeling SORTa small... :-D Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: z/OS Tag Sort
Send the full joblog/sysout from the sort to dfs...@us.ibm.com and we can take a look to see if there are any ways to tune the large sort for you. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 10/13/2011 05:54:04 AM: [image removed] z/OS Tag Sort John Blythe Reid to: IBM-MAIN 10/13/2011 05:54 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List We are sorting a very large file and it's taking about fourteen hours. I remember that there used to be a technique whereby the sort wrote out a list of record addresses and an application program could read the list to access the records in the desired sequence. Does anyone know if there is still a way of doing this ? Thanks. Regards, John. -- John Blythe Reid, Técnico de Sistemas de z/OS y de Sistemas Transaccionales, Barcelona, España. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/03/2011. (returning 10/03/2011)
I am out of the office until 10/03/2011. I am out of the office beginning Friday , 9/30 and returning on Monday, 10/03. My backup is Vicky Vezinaw, vezi...@us.ibm.com Note: This is an automated response to your message Re: OT to my collegues sent on 9/29/11 22:59:20. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: elapsed time for DFSORT
There is not enough information here to determine why the job ran longer. Please send the complete joblog (not just the DFSORT messages) to dfs...@us.ibm.com and we can analyze for you. Also, be sure to send joblogs from multiple days so we can see the variance in elapsed time. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 09/20/2011 12:38:45 AM: [image removed] elapsed time for DFSORT Wang Xiaobing to: IBM-MAIN 09/20/2011 12:39 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hi, Here we have a performance problem on DFSORT. The same DFSORT job run on different day have different elapsed time, the data processed is almost same. Form the job log, we find the different is the longer job show DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK DATA SET SPACE, but our DFSORT job aloways use Hiperspace instead of work data set space. Here is DFSORT log: ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256 ICE128I 0 OPTIONS: SIZE=57313801,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,004),ABCODE=MSG ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX ICE235I 0 OPTIONS: NULLOUT=RC0 ... ICE258I 0 DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK DATA SET SPACE ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 33480 , TRACKS USED: 0 ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES ICE180I 0 HIPERSPACE STORAGE USED = 11628272K BYTES ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES ... And, we checked the SMF TYPE30 for these 2 job, it shows the total connect time(SMF30TCN) obviously increased..and the increased connect time(SMF30DCT ) is only used for SYSOUT DD.. My question is : Q1, why DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK DATA SET SPACE ? and how to correct it ? we do not change the JOB. Q2, In this case, we use Hiperspace instead of work data set space, ICE258I will impace the total connect time ? and cause the elapsed become longer ? Q3, What's the possible root reason increased the total connect time(SMF30TCN) ? Welcome any comments...TIA. WXB. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SLR columns size D_MIGR_SP mbyte gbyte
Wow, this really takes me back. I believe what you have to do is edit the ASM source for the table and recompile/linkedit it. Based on the column names it looks like you are using th eHSM_MIGR_SUM summary table (module DREG0E3D) which is built off of the HSM_MIGRATE log table (module DREG0E2D). The SLR product did make available the assembler source so check your SLR libraries. If can find the source for DREG0E2D and DREG0E3D, the definition for these columns probably contains something like EDIT=F(8). You probably want to change that to EDIT=F(12) and then recompile. You may have to rerun your collect with the updated tables prior to rerunning your reports. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 09/08/2011 04:12:35 AM: [image removed] SLR columns size D_MIGR_SP mbyte gbyte MONTERO ROMERO, ENRIQUE ELOI to: IBM-MAIN 09/08/2011 04:14 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hi team, This time is related to the OLD product (SLR). I am looking for the guides (reference or commands) We have this instruction : SELECT COLUMNS(YEAR MONTH DAY ALLOC_SP D_MIGR_SP T_MIGR_SP T_BCK_SP + NUMDATASET NUM_DASD NUM_TAPE NUM_BKUP) The D_MIGR_SP and T_MIGR_SP results appears with , this means the value cannot fit in the report column. The solution: (not to upgrade to a newer version), could be to divide into 1024 or to increase the columns size, but we have no idea on how to do it. Some clue or guide? Best regards, Enrque Eloi Montero -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/21/2001. (returning 08/22/2011)
I am out of the office until 08/22/2011. I am on vacation beginning Wednesday, 8/17 and returning on Monday, 8/22. My backup is Vicky Vezinaw, vezi...@us.ibm.com Note: This is an automated response to your message Re: Re : A question on Menu's sent on 8/16/11 20:56:39. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/21/2001. (returning 06/13/2011)
I am out of the office until 06/13/2011. I am on vacation Friday, 6/10, returning on Monday, 6/13. My backup is Vicky Vezinaw, vezi...@us.ibm.com Note: This is an automated response to your message jes2 cold Start sent on 6/10/11 3:57:22. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT: DYNAUTO=NO?
I can't get into specifics of our internal algorithms but I can give you some explanations. As for why you are still seeing 6 work data sets allocated for sorts completed in core. You are probably seeing message ICE080I IN MAIN STORAGE SORT. DFSORT tends to be a bit cautious about bypassing allocation of work data sets. We look at the main storage allocated as compared to the file size along with other factors like whether an E15 is present and then only bypass dynamic allocation if we are extremely confident the sort can complete entirely in main storage. Otherwise, we allocate the default number of work data sets which in your case is 6 due to your DYNALOC default of (WOKDA,6). As for those cases where DFSORT will force dynamic allocation even if you have DYNAUTO=OFF. Those cases involve DFSORT's use of data space, Hiperspace or memory object where we believe there may be enough storage to complete the sort without work data sets but we're not overly confident. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 03/07/2011 11:28:57 AM: [image removed] DFSORT: DYNAUTO=NO? Chase, John to: IBM-MAIN 03/07/2011 11:29 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hello, DFSORT Team (in particular), We're on z/OS 1.11, and one of our programmers was cleaning up some JCL by removing explicit SORTWKnn DD statements, among other things. In all job streams he's tested so far, all the sorts were accomplished in core; yet the JES message log shows allocation and deletion of SORTWK01 thru SORTWK06 in every SORT step. What I think the relevant DFSORT installation options we have in place are these: DYNALOC=(WORKDA,6) DYNAUTO=YES DYNSPC=256 There is also this note in the text for DYNAUTO: Even if you specify DYNAUTO=NO, DFSORT can enable automatic dynamic allocation in some cases. Therefore, you should always use DYNALOC to specify the defaults for the device and number of work data sets for use with dynamic allocation. What are (some of) the cases when DFSORT can enable dynamic allocation when DYNAUTO=NO? TIA, -jc- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF data for DFSORT
DFSORT reports cpu time in field ICECPUT. This is basically the step TCB time accumulated between the sort start time and the sort end time. So that time is a subset of the TCB time reported in type 30 records. Also keep in mind that it also means if there are things like user written E15 and E35 exits being used, it's going to include the time spent in those exits so it's not pure DFSORT only cpu time. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 12/01/2010 09:51:31 AM: [image removed] SMF data for DFSORT Michael Hall to: IBM-MAIN 12/01/2010 09:52 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Is there additional information about CPU time for DFSORT in the SMF Type 16 record that is not in the Type 30 step record. In other words, are there any circumstances where CPU time data is written to the Type 16 records and not to the Type 30 records? Do you see step information for DFSORT CPU time in Type 30 records when DFSORT is indirectly invoked from another program? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT chooses not to use Hipersorting for large SORTIN
DFSORT evaluates the available resources and the characteristics of each sort to determine whether or not it would be beneficial to exploit Hiperspace, memory object or dataspace. Even though your system is not paging, it's possible that DFSORT determined there was not enough central strorage available to be of benefit. If you would like more detailed analysis, send a note with your joblog to the DFSORT hotline, dfs...@us.ibm.com and we can look more closely at it. It would also be helpful if you were able to rerun the sort with //SORTDIAG DD DUMMY added to the JCL. This causes DFSORT to write additional diagnostic message to sysout. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 10/06/2010 01:19:53 PM: [image removed] DFSORT chooses not to use Hipersorting for large SORTIN Yifat Oren to: IBM-MAIN 10/06/2010 01:14 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hello All, I have a sort of about 4GB, 15M records of variable length. For some reason DFSORT chooses not to use dataspace/hiperspace or memory objects for this sort for no obvious reason; Parameters are all set corrects (EXPMAX, RES, OLD and HIPRMAX and so on). The system is not paging at all. Only thing not tuned for this SORT (as far as I can see) is that it does not know the AVGRLEN in advance, and wrongly calculated the number of expected records in the SORTIN (it expects 350k instead of 15m, based on the SORTIN LRECL). So, DFSORT volunteered not to use Hipersorting. Looking for answers I found this excerpt from DFSORT Tuning Guide on Hipersorting: When Hipersorting cannot be used, DFSORT uses disk work data sets to store its intermediate data, which is referred to as disk-only mode. Note that Hiperspace-only mode usually provides the best performance when compared to Hiperspace-mixed and disk-only modes. However, this is not always true for Hiperspace-mixed mode when compared to disk-only mode. Due to the additional Hiperspace overhead, the use of disk-only rather than Hiperspace-mixed mode can at times be more advantageous in terms of performance, and therefore DFSORT may choose not to use Hipersorting. So, what are those times; what made DFSORT use disk-only mode? The file size (4GB)? The variable-length? The wrong records number? Any ideas would be appriciated, Yifat -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 10/06/2010)
I am out of the office until 10/06/2010. I am on vacation beginning Wednesday, September 29th, returning on Wednesday, October 6th. If you need immediate assistance contact my backup Vicky Vezinaw (vezi...@us.ibm.com) or my manager, Marc Casad (mdca...@us.ibm.com) Note: This is an automated response to your message Re: IPLTEXT and NUCLEUS dates sent on 9/29/10 0:50:34. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SORT question
I think the real question was how to tell sort to stop reading anymore input records when it reaches the first record that's greater than the selection criteria instead of reading through the rest of the file but not selecting any of those records. I'm not sure we have a way to do that but I'm sure Frank will weigh in shortly. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 09/28/2010 10:41:46 AM: [image removed] Re: SORT question Farley, Peter x23353 to: IBM-MAIN 09/28/2010 10:42 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Richard Pinion Sent: Tuesday, September 28, 2010 10:31 AM To: IBM-MAIN@bama.ua.edu Subject: SORT question I have a large file that is sorted in ascending order by a date field, MMDD. I want to run a sort fields=copy with selection criteria against this file. One of the selection criteria is the date field. I would like the sort to terminate once the date field in the file is greater than my selection date. Is this possible? OPTION COPY OMIT COND=(datepos,datelen,CH,GT,'MMDD') where MMDD is your selected date. If date is not in character format, change CH and date literal to match actual format. You don't need the SORT keyword since this is just a copy. HTH Peter -- This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Query on IMS - OSAM
I believe the discrepancy you are seeing is due to the 8KB block size not allowing you to fully utilize each track. You are probably only getting 6 blocks per track or 49512 bytes since 7 blocks would put you over the track capacity. Now if you muliply your used tracks of 135765 by 49512 that comes pretty close to your 6.15GB calculation. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 09/24/2010 02:14:14 PM: [image removed] Query on IMS - OSAM Veena, Sridhar to: IBM-MAIN 09/24/2010 02:24 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List Hi, I need your help in addressing an OSAM space allocation concern. One of our OSAM database dataset is having allocated space = 150120 tracks and used space = 135765 tracks. The allocation is on 3390 device. If we take one track = 56KB for 3390 device type and do the math, then it looks like our database dataset is already at the allocated space = 8GB ( 150120 * 56KB). This means we cannot allocate more space to the dataset. IMS cannot address OSAMs beyond 8GB. But on verifying the re-org output space stats, the DB dataset is containing 806868 blocks of 8KB each. This suggests the database dataset size to be at 6.15GB (806868 * 8KB). We are confused on which size to consider and whether we can add more space to our OSAM dataset which is at 150120 tracks now (on 3390 device). Thanks Rgds Sridhar K Veena -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 08/17/2010)
I am out of the office until 08/17/2010. I am on vacation August 11-16, returning on Tuesday, August 17. If you need immediate assistance contact my backup Vicky Vezinaw (vezi...@us.ibm.com) or my manager, Marc Casad (mdca...@us.ibm.com) Note: This is an automated response to your message Re: auditor request question sent on 8/11/10 11:59:18. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dynamic sortwork
No DFSORT does not do anything like that. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 07/07/2010 02:20:56 AM: [image removed] Re: Dynamic sortwork Elardus Engelbrecht to: IBM-MAIN 07/07/2010 02:21 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. David Betten wrote: Yes it is correct. We try to allocate all the required space usingn-1 work data sets and we keep that 1 additional to use if the work space requirement is larger than expected. Thanks for kindly helping Radoslaw. I also learned something new today. A question, if you don't mind please, do DFSORT (and perhaps ICETOOL) also use that 1 additional dataset in case there is an ABEND and data needed to be recovered or moved out of the way? Again thanks Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dynamic sortwork
Yes it is correct. We try to allocate all the required space using n-1 work data sets and we keep that 1 additional to use if the work space requirement is larger than expected. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 07/06/2010 11:21:25 AM: [image removed] Dynamic sortwork R.S. to:. IBM-MAIN 07/06/2010 11:22 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I observed that DFSORT does use n-1 dynamic sortwk files. For n=4 only 3 files have non-zero space. Fourth is empty (only VTOC entry). It's unrelated to number of files or space constraints. For the same data to be sorted one can increase n and still gets approx. the same total number of tracks, and one of the files is empty. Q1: Is the above correct? Q2: What is the purpose of the empty file? -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sąd Rejonowy dla m. st. Warszawy XII Wydział Gospodarczy Krajowego Rejestru Sądowego, nr rejestru przedsiębiorców KRS 025237 NIP: 526-021-50-88 Według stanu na dzień 01.01.2009 r. kapitał zakładowy BRE Banku SA (w całości wpłacony) wynosi 118.763.528 złotych. W związku z realizacją warunkowego podwyższenia kapitału zakładowego, na podstawie uchwały XXI WZ z dnia 16 marca 2008r., oraz uchwały XVI NWZ z dnia 27 października 2008r., może ulec podwyższeniu do kwoty 123.763.528 zł. Akcje w podwyższonym kapitale zakładowym BRE Banku SA będą w całości opłacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 07/12/2010)
I am out of the office until 07/12/2010. I am on vacation July 5-9, returning on Monday, July 12. If you need immediate assistance contact my backup Vicky Vezinaw (vezi...@us.ibm.com) or my manager, Marc Casad (mdca...@us.ibm.com) Note: This is an automated response to your message Re: GML (was Unix systems and Serialization mechanism). sent on 7/2/10 22:09:06. This is the only notification you will receive while this person is away. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: supported releases of DFSORT
They coincide with the z/OS EOS dates. DFSORT V1R5 is the DFSORT release for z/OS 1.5 thru 1.9. So the EOS for z/OS 1.9 is the EOS for DFSORT V1R5. I don't know what that date is off the top of my head. I can look it up if you need it. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 07/02/2010 09:12:02 AM: [image removed] supported releases of DFSORT Jim McAlpine to: IBM-MAIN 07/02/2010 09:12 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I can't find DFSORT mentioned on the IBM Software Support Lifecycle site and the DFSORT site doesn't seem to mention EOS dates. Can someone point me to EOS dates for DFSORT. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: supported releases of DFSORT
Correct. Thanks, so V1.5 is the oldest supported release, correct. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT and SORTWK space
Your assumptions are correct. You can increase the total amount of work space possible by increasing the number of work data sets that DFSORT dynamically allocates. This is by using the n value in the DYNALOC installation default or the DYNALLOC run time options. For example OPTION DYNALLOC=(SYSDA,8) Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 06/20/2010 01:30:38 AM: [image removed] DFSORT and SORTWK space R.S. to:. IBM-MAIN 06/20/2010 01:31 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. If you omit SORTWKnn DDnames then - assumed default values used - DFSORT will use 3 datasets, in LARGE format. Every sort work dataset is single volume, so the limit for default sort work space depends on size (and free space) of work volumes. Q: Am I right with the above? For example, I have several mod-54 work volumes, so my sort work space limit is approximately 3x65500 cylinders. Further question: What if volumes are framgmented? LARGE PS has limit of 16 extents, so the volume fragmentation could lead to smaller maximum sizes - is it true? Regards -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sąd Rejonowy dla m. st. Warszawy XII Wydział Gospodarczy Krajowego Rejestru Sądowego, nr rejestru przedsiębiorców KRS 025237 NIP: 526-021-50-88 Według stanu na dzień 01.01.2009 r. kapitał zakładowy BRE Banku SA (w całości wpłacony) wynosi 118.763.528 złotych. W związku z realizacją warunkowego podwyższenia kapitału zakładowego, na podstawie uchwały XXI WZ z dnia 16 marca 2008r., oraz uchwały XVI NWZ z dnia 27 października 2008r., może ulec podwyższeniu do kwoty 123.763.528 zł. Akcje w podwyższonym kapitale zakładowym BRE Banku SA będą w całości opłacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: icetool help with short fields
Frank will correct me if I'm mistaken but I believe you might need the
VLSCMP option. This will cause DFSORT to pad short variable
length INCLUDE/OMIT compare fields with binary zeroes.
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email: bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 03/01/2010
12:07:20 PM:
[image removed]
Re: icetool help with short fields
Bri P
to:
IBM-MAIN
03/01/2010 12:08 PM
Sent by:
IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu
Please respond to IBM Mainframe Discussion List.
Well the message is telling you that you're expecting a field at
least 28 bytes long (because of the 19,10,CH statement) but you've
only read in a record of 26 bytes.
That does seems a bit strange. I mean, that date field is surely
either going to be there in its entirety or not at all, not just
have two bytes missing?
I can't see anything wrong with the SORT or ICETOOL statements.
Is your source IRRDBU00 extract ok? (browse it - it's nearly all
readable text data)
Maybe also write the TMP0001 extract to a non-temporary file and
examine that too.
Brian
-Original Message-
From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
Behalf Of larry macioce
Sent: 01 March 2010 16:14
To: IBM-MAIN@bama.ua.edu
Subject: icetool help with short fields
I was graciously given some jcl to do a sort of an unloaded racf db.
The problem is I am getting:
ICE218A 1 26 BYTE VARIABLE RECORD IS SHORTER THAN 28 BYTE MINIMUM FOR
FIELDS
the jcl is:
//TOOLIN DD *
SORTFROM(INDD) TO(TEMP0001) USING(ULDF)
DISPLAY FROM(TEMP0001) LIST(PRINT) -
PAGE -
TITLE('LIST OF IDS') -
DATE(4MD/) -
TIME(24:) -
BLANK -
ON(10,8,CH) HEADER('USER ID') -
ON(79,20,CH) HEADER('USER NAME') -
ON(100,8,CH) HEADER('DEFAULT GROUP') -
ON(19,10,CH) HEADER('CREATED')
//INDD DD DISP=SHR,DSN=SYS1.RACF.RPT
//TEMP0001 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(20,5,0)),
// UNIT=SYSDA
//ULDFCNTL DD *
SORT FIELDS=(10,8,CH,A)
INCLUDE COND=(5,4,CH,EQ,C'0200',AND,
19,10,CH,GE,C'2008-07-01',AND,
19,10,CH,LE,C'2009-06-30') OPTION VLSHRT
As you can see I was given the vlshrt (didnt know what it was, looked it
up)
option so now I am confused.
Thanks
mace
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SORTWK files
Larry, Sounds good. Also, don't forget about the DFSORT hotline (dfs...@us.ibm.com) if you have questions or need help debugging a problem as you develop your solution. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/15/2010 09:28:44 AM: [image removed] Re: SORTWK files Larry Crilley to: IBM-MAIN 02/15/2010 09:33 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I would not read a single record and pass to each of the sorts. I will hope to fill multiple buffers with many records. For example, why not get 5 buffers (or more) and each buffer is 1 Meg (or larger) in size. Stash as many records as possible in each buffer and have each sort pull the records from these buffer areas. These buffer areas could be above the bar. I'll have to do a little buffer management and know when each sort is done with each buffer, but all in all, not too difficult. -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of David Betten Sent: Friday, February 12, 2010 8:48 PM To: IBM-MAIN@bama.ua.edu Subject: Re: SORTWK files That's a very good point. You need to take into account how much virtual storage is available when running parallel sorts in the same address space. You could limit each sort by passing a DSA value to limit DFSORT's dynamic storage adjustment from allocating more storage than is available. This is also another good reason to run with as few work data sets as possible since they each require virtual storage for control blocks, buffers, etc. I'm also starting to wonder just how much elapsed time would be saved by using this strategy of having a program read the input and then pass to multiple sorts. With today's world of PAV and high speed devices/channels, is this really going to be much faster than having each read the input? I guess testing will bare that out. This strategy is similar to one we used to see employed with BatchPipes. You'd have one reader job which then passed the records to multiple concurrent jobs through pipes. What I tended to find, is that you end up serializing the reads and the overall result is increased elapsed time. Basically they are all going to read at the pace of the slowest sort. For example you pass records to 4 concurrent sorts. SORT1 will get record 1 and then wait until that's been passed to SORT2, SORT3, SORT4 before it gets record 2. Then it waits again for record 3 and so on and so on. These parallel sorts are probably going to be doing a lot of waiting for input. You can probably minimize this with some sort of large buffer area and wait/post logic but my guess is that will be some complex code. IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/12/2010 06:37:47 PM: [image removed] Re: SORTWK files Blaicher, Chris to: IBM-MAIN, 02/12/2010 06:38 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I just noticed the following in one of your postings. If I want to subtask multiple sorts (same input file), If you are going to run multiple sorts concurrently in the same address space, things can get interesting. NOTE: You may want to confirm what I say with your sort vendor. Sorts like resources, mainly memory. They also think they are the only thing running in the address space, or that whatever else is running will not get more memory once the sort has started. This means that starting multiple sorts can have a very negative impact on the sorts started first. Running multiple sorts will often times get you an 878 type abend, especially if your REGION is not REGION=0M. If all the sorts are similar in size and characteristics, then maybe a simple thing as passing the proper parameters in SORTPARM may do the trick. If you have sorts of different sizes and different characteristics and run them all with a common set of over-rides, then some are getting more than they need and others are probably getting less. This is not as simple as it sounds. To put it in perspective, I have 25 years of experience in trying to get it right. Of course, I am trying to balance anywhere from 1 to 48 concurrent sorts of different types and sizes. You'll get it to run, but getting it to run the best it can is not easy. Chris Blaicher Phone: 512-340-6154 Mobile: 512-627-3803 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives
Re: SORTWK files
For DFSORT, we generally recommend 4 work data sets. I have done testing with varied numbers of work data sets and not seen much variance in performance until you start allocating very large numbers of work data sets for small sorts.Also, I would recommend that you allocate total space of about 10% more than the file size. So for your 100 cylinder data set, you'd want to allocate 4 work data sets of about 28 cylinders each. Also, we generally don't recommend allocating your own work data sets and instead let DFSORT's dynamic allocation handle it. DFSORT can take into account how much central storage is going to be used and adjust the work data set space allocations accordingly. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/12/2010 09:13:53 AM: [image removed] SORTWK files Larry Crilley to: IBM-MAIN 02/12/2010 09:15 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I want to dynamically allocate my own sort work files. If I know how large my input file is (I can read the catalog), then what would everyone recommend for the allocation of the sort work files? How many datasets are optimal? If my input file is 100 cylinders, for example, would you allocate 3 sort work files CYL(35,0), 4 sort work files CYL(25,0), 8 sort work files.? Larry Crilley Dino-Software Corporation 800.480.DINO 412.366.3566 outbind://92-0C19A532C55CD94285A5E250EF9EC5A7445E7B00/www.dino-soft ware.com www.dino-software.com blocked::blocked::http://www.dino-software.com/trex_factsheet.php T-REX - Superior ICF catalog mgmt with full Tape support and HSM Auditing blocked::blocked::http://www.dino-software.com/reorgadon_factsheet.php REORGadon - First ever online REORG for HSM blocked::blocked::http://www.dino-software.com/teradon_factsheet.php TERADON - First ever REPRO MERGECAT While-OPEN blocked::blocked::http://www.dino-software.com XTINCT - Secure DASD/TAPE data eradication blocked::blocked::http://www.dino-software.com/rtd_factsheet.php RTD - DASD Real Time Defrag blocked::blocked::http://www.dino-software.com/dal_details.php DAL - DINO healthcheck Analysis service for Legato blocked::blocked::http://www.dino-software.com/sentinel_factsheet.php SENTINEL - Real-time FTP Management. All Secure, all the time. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SORTWK files
DFSORT tells me: Generally, DFSORT can automatically determine the input file size. However, in a few cases, such as when an E15 supplies all of the input records, Actually it's when E15 supplies all of the input records AND no file size information is provided to DFSORT. If you pass a file size and record length information, then DFSORT will have the information it needs to calculate the correct amount of work space as well other resources like main storage and such. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SORTWK files
Many moons ago (maybe DFSORT 1.7) the recommended total SORTWK space was 1.7 times the input file size. I've noted how using Hiperspace and Dataspace reduces the sortwk space used, and I'm wondering if your 10% recommendation is based on Sort being able to use an optimal amount of memory. Actually our recommendation has always been 1.3 times the file size, not 1.7. But as I've been looking at more and more customer data, I'm finding that 1.1 is usually sufficient. And given the size of today's sorts, 1.3 can cause quite a bit of additional space to be allocated. If so, would you recommend a larger value for SORTWK space on memory constrained system, or where customers have hobbled sort's memory in the ways that are often discussed on this list? In other words, is 110% good enough if sort chooses the smallest dynamic memory options. The 1.1 times the file size recommendation assumes you are not going to use any central storage for sorting in memory. So it's the worst case. This is one of the reasons we recommend using DFSORT's dynamic allocation since it takes into account the expected central storage exploitation and adjusts the disk work space allocation accordingly. Some questions on your SORTWK performance tests: Does CFW on/off affect the impact when many SORTWK datasets are allocated (I'm assuming you mean 16 or more), and does the difference between 4 and many SORTWK change in any way when there is an intermediate merge? Four is really just a good starting point. I couldn't make a blanket rule of thumb recommendation to use x if CFW is on and y if CFW is off. There are so many other factors like the number of controllers, cache sizes, I/O contention, etc. And a query: is the recommendation of 4 SORTWK good up to a particular size. I'm interested in that point because four SORTWK would be limited to about 17K CYLS total. Again, 4 is just a good starting point. I think the size of the sort and the size and number of volumes in the work pool would really factor in how many work data sets you want/need to allocate. Also, DFSORT supports large format data sets for work data sets so we can actually use much more than 17K cylinders with 4 work data sets. DFSORT's dynamic allocation automatically allocates the work data sets as large format. Ron -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of David Betten Sent: Friday, February 12, 2010 6:39 AM To: IBM-MAIN@bama.ua.edu Subject: Re: [IBM-MAIN] SORTWK files For DFSORT, we generally recommend 4 work data sets. I have done testing with varied numbers of work data sets and not seen much variance in performance until you start allocating very large numbers of work data sets for small sorts.Also, I would recommend that you allocate total space of about 10% more than the file size. So for your 100 cylinder data set, you'd want to allocate 4 work data sets of about 28 cylinders each. Also, we generally don't recommend allocating your own work data sets and instead let DFSORT's dynamic allocation handle it. DFSORT can take into account how much central storage is going to be used and adjust the work data set space allocations accordingly. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/12/2010 09:13:53 AM: [image removed] SORTWK files Larry Crilley to: IBM-MAIN 02/12/2010 09:15 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I want to dynamically allocate my own sort work files. If I know how large my input file is (I can read the catalog), then what would everyone recommend for the allocation of the sort work files? How many datasets are optimal? If my input file is 100 cylinders, for example, would you allocate 3 sort work files CYL(35,0), 4 sort work files CYL(25,0), 8 sort work files.? Larry Crilley Dino-Software Corporation 800.480.DINO 412.366.3566 outbind://92-0C19A532C55CD94285A5E250EF9EC5A7445E7B00/www.dino-soft ware.com www.dino-software.com blocked::blocked::http://www.dino-software.com/trex_factsheet.php T-REX - Superior ICF catalog mgmt with full Tape support and HSM Auditing blocked::blocked::http://www.dino-software.com/reorgadon_factsheet.php REORGadon - First ever online REORG for HSM blocked::blocked::http://www.dino-software.com/teradon_factsheet.php TERADON - First ever REPRO MERGECAT While-OPEN blocked::blocked::http://www.dino-software.com XTINCT - Secure DASD/TAPE data eradication blocked::blocked::http://www.dino-software.com/rtd_factsheet.php RTD
Re: SORTWK files
That's a very good point. You need to take into account how much virtual storage is available when running parallel sorts in the same address space. You could limit each sort by passing a DSA value to limit DFSORT's dynamic storage adjustment from allocating more storage than is available. This is also another good reason to run with as few work data sets as possible since they each require virtual storage for control blocks, buffers, etc. I'm also starting to wonder just how much elapsed time would be saved by using this strategy of having a program read the input and then pass to multiple sorts. With today's world of PAV and high speed devices/channels, is this really going to be much faster than having each read the input? I guess testing will bare that out. This strategy is similar to one we used to see employed with BatchPipes. You'd have one reader job which then passed the records to multiple concurrent jobs through pipes. What I tended to find, is that you end up serializing the reads and the overall result is increased elapsed time. Basically they are all going to read at the pace of the slowest sort. For example you pass records to 4 concurrent sorts. SORT1 will get record 1 and then wait until that's been passed to SORT2, SORT3, SORT4 before it gets record 2. Then it waits again for record 3 and so on and so on. These parallel sorts are probably going to be doing a lot of waiting for input. You can probably minimize this with some sort of large buffer area and wait/post logic but my guess is that will be some complex code. IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/12/2010 06:37:47 PM: [image removed] Re: SORTWK files Blaicher, Chris to: IBM-MAIN, 02/12/2010 06:38 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I just noticed the following in one of your postings. If I want to subtask multiple sorts (same input file), If you are going to run multiple sorts concurrently in the same address space, things can get interesting. NOTE: You may want to confirm what I say with your sort vendor. Sorts like resources, mainly memory. They also think they are the only thing running in the address space, or that whatever else is running will not get more memory once the sort has started. This means that starting multiple sorts can have a very negative impact on the sorts started first. Running multiple sorts will often times get you an 878 type abend, especially if your REGION is not REGION=0M. If all the sorts are similar in size and characteristics, then maybe a simple thing as passing the proper parameters in SORTPARM may do the trick. If you have sorts of different sizes and different characteristics and run them all with a common set of over-rides, then some are getting more than they need and others are probably getting less. This is not as simple as it sounds. To put it in perspective, I have 25 years of experience in trying to get it right. Of course, I am trying to balance anywhere from 1 to 48 concurrent sorts of different types and sizes. You'll get it to run, but getting it to run the best it can is not easy. Chris Blaicher Phone: 512-340-6154 Mobile: 512-627-3803 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: why compression costs additional I/O?
generally all of it probably mean that using DFSORT for compressed datasets is not good idea. I'm not sure I would agree with a general statement such as that. First. There is a cpu overhead associated with compression and it effects ALL applications, not just sort. The overhead is generally higher for write than it is for read. In some cases, that overhead can be offset by reduced data transfer but that depends on how well the data compresses. You also need to look at how the data set is used. If it's written once but read many times, then you may get enough benefit on all those reads to warrant the negative impact on the write. Second, higher EXCPs does not necessarily mean higher I/Os. For BSAM, buffers are used to store the blocks which are then chained together in single I/Os. So the increase in I/Os is likely much smaller than the increase in EXCPs. Thirdly, you may want to consider multiple stripes so that data is transferred in parallel. This won't reduce the I/Os but it would allow multiple I/Os to be done in parallel and reduce elapsed time. I've never really considered compression as a means of improving performance. I've heard all the arguments about less data being transferred but in all my years of batch tuning I never really saw that great an impact to offset the cpu cost. To me, compression is great for avoiding out of space conditions and managing very large files. When performance is the sole concern, I've always recommended extended format with multiple stripes but not compressed. Of course that requires that. you have the disk space available to support storing the large data sets! Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/27/2010 10:23:22 AM: [image removed] Re: why compression costs additional I/O? Pawel Leszczynski to: IBM-MAIN 01/27/2010 10:26 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. Hi Yifat, Thanks for answer - you are right! - I 've checked in joblog: for compressed output: 0 SORTOUT : BSAM USED but for non-compressed output: SORTOUT : EXCP USED generally all of it probably mean that using DFSORT for compressed datasets is not good idea. Regards, Pawel On Wed, 27 Jan 2010 15:55:24 +0200, Yifat Oren yi...@tmachine.com wrote: Hi Pawel, The reason is the sort product can not use the EXCP access method with the compressed data set and instead chooses BSAM as the access method. The EXCP access method usually reads or writes on a cylinder (or more) boundary while BSAM, as its name suggests, reads or writes block by block. Hope that helps, Yifat Oren. -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Pawel Leszczynski Sent: Wednesday, January 27, 2010 12:56 PM To: IBM-MAIN@bama.ua.edu Subject: why compression costs additional I/O? Hello everybody, Recently we are reviewing our EndOfDay jobs looking for potential performance improvements (reducing CPU/elapsed time). We have several jobs sorting big datasets where output is SMS-compressible (type: EXTENDED) datasets. When we compare such sorting with sorting on non-compressible output we can see this: EXCP TCB SRB el.time TESTXWP5 STEP110 00 757K 3.51.709.01 -- w/o compression TESTXWP5 STEP120 00 1462K 3.62 2.89 10.45 -- w. compresion We guess that big SRB in (2) goes for compression (that we understand - we probably quit compression at all), but we don't understand 2 times bigger EXCP in second case. Any ideas will be appreciated, Regards, Pawel Leszczynski PKO BP SA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at
Re: DFSORT memory used paging
Good points Kees. The original question didn't indicate a paging problem, just an increase in page DS use and demand paging. So yes, tuning the paging subsystem is certainly a valid approach. I was speaking mainly from a DFSORT perspective on ways to minimize paging. Usually a small amount of demand paging isn't a problem, but then again it depends on what workload is doing it. IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/15/2010 03:47:26 AM: Dave, I would have expected you to also mention a different approach: check the paging configuration. During the years I have become quite confident about the intelligence DFSORT has developped to use the available system resources, without causing problems in the system. One of the presumptions is the installation has a adequate paging configuration. We had a similar situation lately where we had paging performance problemns during DFSORT jobs. We have ample storage and the large DFSORT jobs decided to take a couple of GB central storage for its sort. This caused a large page out activity (like you described above) with I/O problems. After investigating the situation, the conclusion could only be that either DFSORT miscalculated the amount of available storage *or* the paging subsystem was not able to handle a substantial page-out load. We looked at our paging configuration and the spreading of the volumes over the internal ESS ranks and concluded that it contained several hotspots. We reconfigured the paging volumes and never saw the problem again. Everybody has read many years ago the recommandation for his/her paging configuration to spread it over as many volumes, channels and control units as possible, said yes, of course, and continued with other things. A under-optimal paging configuration works well most of the time except in special situations like large dumps and yes, large DFSORTs. I think that also in Rez's case, both sides of the problem should be examined and if the paging configuration is insufficient *and* he is not able to correct this, the other option is to limit DFSORT's use of storage. Kees. ** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 ** -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT memory used paging
DFSORT memory used paging R Hey to: IBM-MAIN 01/14/2010 09:20 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. Hi Frank/List, My client has some very big SORT jobs. When a few run at the the same time, the page DS USE% increases (demanding paging). This is likely due to DFSORT creating large memory objects, Hiperspaces or Dataspaces for sorting in memory and that's causing pages to be stolen from other workloads which then get paged out. Q1- How do I make sure BIG sort jobs don't use too many frames, when they are hard to get? DFSORT has installation defaults that can limit the storage used by all concurrent sorts. EXPOLD controls the amount of old (not recently referenced) frames that are considered in DFSORT's calculations of storage available for sorting in memory. The shipped default is MAX. You can change this if you want to limit that number of old frames DFSORT uses which in turn would reduce the page stealing and demand paging. For example, if you set EXPOLD=0, then DFSORT would only consider available frames in its calculations of storage availabe for memory objects, Hiperspaces and Dataspaces. Q2- Given the sort options we use, is it true that SORT will use TMAXLIM size of storage, then if data is too big, it will increase the storage use to max of DSA size? So it never goes over DSA size? This is correct. Q3- If I get ICE199I 0 MEMORY OBJECT STORAGE USED = 3463M BYTES it means 3.4GB of data was sorted in memory, right? Correct. Q4- If YES to both Q2/Q3, then how is 3GB of data is sorted in 64M of memory (our DSA) If, not, how does it work? DFSORT will use up to 64MB of virtual storage for buffers, control blocks, etc. It can also create Hiperspaces or memory objects to be used as intermediate storage for sorting the data. The intermediate storage is limited by DFSORT installation defaults such as EXPMAX, EXPOLD, HIPRMAX, MOSIZE, DSPSIZE, etc. It's also limited by the amount of available storage at the time the sort executes. Q5- We use SIZE=MAX, but the job has SIZE=32259547 (it can be different for another run). Is this because the size of data being sorted determines the actual SIZE used? Yes, DFSORT's Dynamic Storage Adjustment (DSA) alogorithm calculates the optimal amount of main storage for a sort. The DSA installation default is a limit. So DFSORT will use minimum of the calculated value or the DSA limit. TIA, Rez Our sort option are: MODULE ICEAM1 - - - - - - - - - - - ARESALL0 ARESINVNOT APPLICABLE - - - - - - - - - - - DSA64 - - - - - - - - - - - DYNALOC(SYSALLDA,3) * (SYSDA,4) - - - - - - - - - - - EXPMAX MAX EXPOLD MAX EXPRES 0 - - - - - - - - - - - HIPRMAXOPTIMAL - - - - - - - - - - - MAXLIM 1048576 MINLIM 450560 MOSIZE MAX - - - - - - - - - - - OVERRGN65536 RESALL 4096 RESET YES RESINV NOT APPLICABLE - - - - - - - - - - SIZE MAX - - - - - - - - - - TMAXLIM6291456 - - - - - - - - - - VIONO joblog had: ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED ICE088I 5 PAK328 .SORT305 . , INPUT LRECL = 650, BLKSIZE = 27950, TYPE = FB ICE093I 0 MAIN STORAGE = (MAX,32259547,32259547) ICE156I 0 MAIN STORAGE ABOVE 16MB = (32122707,32122707) ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256 ICE128I 0 OPTIONS: SIZE=32259547,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=AB END,MS GDDN=SYSOUT ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC= (SCRATCH ,004),ABCODE=MSG ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DS A=64 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX ICE235I 0 OPTIONS: NULLOUT=RC0 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN ICE750I 0 DC 3576062750 TC 0 CS DSVVV KSZ 36 VSZ 36 ICE752I 0 FSZ=5501635 RC IGN=0 E AVG=652 0 WSP=4658983 C DYN=0 0 ICE751I 1 BA-K22788 BB-K24705 BC-K24705 E8-K90013 ICE090I 0 OUTPUT LRECL = 650, BLKSIZE = 27950, TYPE = FB (SDB) ICE080I 0 IN MAIN STORAGE SORT ICE055I 0 INSERT 0, DELETE 0 ICE054I 0 RECORDS - IN: 5501623, OUT: 5501623 ICE134I 0 NUMBER OF BYTES SORTED: 3576054950 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0 ICE199I 0 MEMORY OBJECT STORAGE USED = 3463M BYTES
Re: DFSORT memory used paging
Recommend you start at the IBM DFSORT home-page where there are several documentation and supplemental technical references on DFSORT and memory usage. Scott Barry SBBWorks, Inc. Suggested Google advanced search argument on this topic: dfsort memory usage site:ibm.com Good point Scott. I should've included that in my original post. There's also the DFSORT Tuning Guide which is very helpful. Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Sort Capacity Exceeded thought / question
John, The answer is that it's a little of both... technical reason and we didn't think of it. We are looking at ways to improve our dynamic allocation of work data sets to provide some additional flexibility. So it is on our list of things we'd like to improve on. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 12/21/2009 01:57:44 PM: [image removed] Re: Sort Capacity Exceeded thought / question McKown, John to: IBM-MAIN 12/21/2009 02:02 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Lizette Koehler Sent: Monday, December 21, 2009 12:47 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Sort Capacity Exceeded thought / question John, What is your current parameter for dynamic SortWKxx set at? Maybe you just need to increase that number? Or if you have one-off types, provide them with the SORT Over ride that could let them allow for more dynamically allocated sortwk. Lizette We have DYNALLOC set to 31. But this is in an ICEAM2 environment (internal sort). Apparently the application does not pass an accurate count to SORT. And, from what I can see, it does multiple internal sorts. I did state a JCL error, but the error was not having an SORTWKnn statements in the JCL at all. But that is not the prime reason for my inquiry. I am just curious if this facility is not in sort for a technical reason, a business reason, or a we didn't think of that reason. I do know how to fix it - we added a lot of large SORTWKnn DD statements. -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * (817)-961-6183 cell john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid- West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: IRLM timeout after IPL
Did you change the size of the IRLM lock structure? This is going back a ways but I recall having a problem when the IRLM lock structure was too large, it caused a loop when calculating the number of entries. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 09/29/2009 12:45:22 PM: [image removed] IRLM timeout after IPL Miklos Szigetvari to: IBM-MAIN 09/29/2009 12:46 PM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. Hi If someone has got this: After an IPL we got : DXR133I MRLM001 TIMEOUT DURING GLOBAL INITIALIZATION WAITING FOR NOEVENT and unable to start DB2 MQM Have opened a PMR, but maybe someone has seen this. -- Miklos Szigetvari Development Team ISIS Information Systems Gmbh tel: (+43) 2236 27551 570 Fax: (+43) 2236 21081 E-mail: miklos.szigetv...@isis-papyrus.com Info: i...@isis-papyrus.com Hotline: +43-2236-27551-111 Visit our Website: http://www.isis-papyrus.com --- This e-mail is only intended for the recipient and not legally binding. Unauthorised use, publication, reproduction or disclosure of the content of this e-mail is not permitted. This email has been checked for known viruses, but ISIS accepts no responsibility for malicious or inappropriate content. --- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: help with smf
SMF type 15 is written for data sets that are opened for OUTPUT, UPDAT, INOUT or OUTIN processing. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 08/04/2009 09:05:46 AM: [image removed] help with smf larry macioce to: IBM-MAIN 08/04/2009 09:07 AM Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu Please respond to IBM Mainframe Discussion List. I have a ps file that is getting changed and I need to find the job doing it.I need to know which smf records I need to pull to get the info. I've looked through the smf record types but it isn't jumping out at me. Any help is greatly appreciated thanks Larry -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT on z/OS 1.10 problem
John, I think you should open a PMR and we'll research the cause of the error. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 06/15/2009 11:02:59 AM: Well, I finally found something. If I put in a MOSIZE=0 on the OPTION statement in DFSPARM, then the problem goes away. Sorry, but I'm just not having a good week (too many weekend problems). -- John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Using FTP to send loadlib
I usually just terse with TRSMAIN, ftp the tersed file, then unterse with TRSMAIN. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 04/27/2009 01:32:25 PM: -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Kurt Eastwood Sent: Monday, April 27, 2009 10:32 AM To: IBM-MAIN@bama.ua.edu Subject: Using FTP to send loadlib Hello, =A0 I have been digging through the OS/390 TCP/IP OE: User's Guide and have not= been able to get this to work. =A0 Is it possible to send a loadlib from 1 mainframe to another mainframe usin= g FTP, either batch or interactive? =A0 If so, would anyone be willing to share some batch jcl with me? =A0 Thanks, Kurt=0A=0A=0A SNIP I think you will have to unload it using IEBCOPY to a flat file, then send that as BIN. After receiving it, use IEBCOPY to load the PDS. OR, you can use TSO XMIT to a file, FTP as BIN, then point TSO RECEIVE to the file... Regards, Steve Thompson -- Opinions expressed by this poster may not reflect those of poster's employer. -- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF reporting question
I'm not that familiar with it but the RMF Spreadsheet Reporter might povide you with some z/OS performance reporting. I'm pretty sure it comes with RMF so it probably meets your requirement of being something you already have. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/26/2009 11:07:43 AM: I realize that most people likely use SAS for SMF reporting. We did this in the past. Management has declared that SAS on the z is simply too expensive. They also declared that installing the Windows version on our main user's PC is also too expensive. End of discussion on that point. So, what else could be used? It must either be something that we already have, such as COBOL, EasyTrieve Plus (without the SMF add on), REXX, HLASM, ... . We cannot spend any hard money on this. Oh, it would be nice if if were very CPU efficient because we just downgraded our z9BC from a V02 to a T02 in order to save on software costs. And it must be such that doing ad hoc requests can be responded to quickly. Yes, I know, give me the world, but don't spend any money. I am actually looking at downloading the raw SMF data to a Linux box (my desktop) using BINary and SITE RDW. I did this for last week's SMF and had about 14Gib of data. I know how to read this with Java. This may actually be what I end up looking at doing. But I am the only person in my group who is even mildly Java literate. The main performance person is not. And he doesn't have access to my PC anyway. Of course, that is one reason that I'm looking at Java. I have written Java in the past (minor application) which truly was run anywhere. At least it ran, as compiled on the Linux box, on 32 bit Linux/Intel, 64 bit Linux/Intel, Mac OS/X, 32 bit Windows, and on the z. I just transferred the jar file and ran it. Any thoughts or commiserations appreciated. -- John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT statement sequence
It is FALSE. All of the control statements are read at initialization and the include fields are evaluated during the input phase of the sort to select only the records that meet the criteria. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 02/09/2009 07:52:51 AM: Good day Where is it documented that the sequence of sort control statement does play a role in performance? My searches on Google, IBM-MAIN and DFSORT couldn't produce something 'hard and fast'... My gut feeling is that INCLUDE COND=(1,8,CH,EQ,C'ABCDEFGH') SORT FIELDS=(1,8,CH,A) is better over this SORT FIELDS=(1,8,CH,A) INCLUDE COND=(1,8,CH,EQ,C'ABCDEFGH') for very LARGE input. Is this true or false? (If false, I can always do a COPY with INCLUDE and then do my SORT FIELDS) Thank you in advance. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT statement sequence
Thanks for your kind answer. Does that means that AFTER reading all the matching input records, that subset is THEN sorted? Correct. As each record is processed, the INCLUDE criteria is evaluated. If a the record does not meet the INCLUDE criteria it is discarded and not included in the sort. The reason is I have to select a wide variety of different inputs toproduce a subset and THEN sort that lot for reporting. My goal: to have as fewbytes to sort after selecting what I need with one pass of the input dataset. I wonder, if I do all my COPY and using INCLUDE statements and then do this two stage method: COPYFROM(INPUT) TO(TEMP1) USING(dd with INCLUDE) COPYFROM(TEMP1) TO(TEMP2) USING(dd with SORT) Will this work better for large input datasets? This is one of those it depends types of situations. If a large percentage of the records are likely to be included, I think you are fine with just doing 1 sort with the include statements. If a small percentage are likely to be included, you may be better off doing the copy with INCLUDE followed by the sort. The reason I say this is that when sorting with INCLUDE, DFSORT has no way of knowing how many records will match the criteria. Therefore, we need to allocate resources (main storage, work space, etc.) under the assumption that all records will be included. So you may have some wasted resources if only a small percentage of the records are actually included. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: hiperspace, DSNUTILB and df/sort
I'm not entirely sure I understand your question but I'll try to explain how DFSORT uses Hiperspace and maybe that will help. As part of DFSORT initialization, the characteristics of the sort are evaluated along with available resources and a decision is made about what type and how much storage to use. If a decision was made to use Hiperspace storage than a Hiperspace is created. At this point the Hiperspace is empty which may be why you see it as swapped out. As the DB2 utility passes input records to DFSORT, they are written to the Hiperspace. I'm guessing this filling of the Hiperspace might be what is looking like page ins to you. Once all input records have been processed, DFSORT then begins to read the records back from the Hiperspace and pass them back to the DB2 Utility in sequence. Normally we use RELEASE=YES so that the pages are released as they are read. At completion, DFSORT then deletes the Hiperspace. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com 1-301-556-0943 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 01/23/2009 01:32:12 PM: We are trying to understand how hiperspace is being used. z/os 1.9. Using DSNUTILB and it's invocation of df/sort, it appears that when the hiperspace is initially created, it is immediately swapped out, then demand paged back in. When I look at type30 records, or in MON3 we see pagins, but never any page outs, nor any swap outs during the step duration. in MON3, we see sometimes 1K's worth of pages in DIV status, but again, we never see any page/swap outs. What am I missing or where should I be looking ? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Is there a tool to put SMF data into DB/2?
I believe IBM's Tivoli Decision Support for z/OS collects SMF data into DB2 tables for analysis and reporting. I think this tool used to be called EPDM. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: bet...@us.ibm.com DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu wrote on 12/17/2008 07:30:57 AM: I was poking around in the manuals and saw some things about possibly putting the SMF data related to RACF into DB/2 tables, and it got me wondering. Is there a tool that will convert *all* SMF record types and put them into DB/2 tables? That would be awesome if there were. David Logan Manager of Product Development, Pitney Bowes Business Insight http://centrus.com/ http://centrus.com W: (720) 564-3056 C: (303) 818-8222 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Finding the correct IEASYSxx member
I think if you enter D IPLINFO from the console it gives you the IEASYS list. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 10/15/2008 02:55:21 PM: How can I trace back and find out which IEASYSxx member the system was IPL'ed with? If you don't key in a member name at IPL time doesn't the system pick IEASYS00? Thanks. _ LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately, then delete this message and empty from your trash. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Opinions? Subset input with DFSORT before processing in REXX?
I would use DFSORT to extract the 0200 records and have your REXX read the smaller file. Your REXX is probably using EXECIO and DFSORT will be able to read through that entire file faster and use much less cpu time. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 05/23/2008 10:25:07 AM: I would, definitely. You can do some Wonderful Things with DFSORT/ICETOOL these days. The SPLICE operator is particularly wonderful for RACF database unloads as you can join up base userid records with their segments, so you can then handle single records at a time in a report instead of having to go doing subsequent reads for the segments. I love Frank Yaeger. He's a God! ;-) Brian - Email sent from www.virginmedia.com/email Virus-checked using McAfee(R) Software and scanned for spam -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Long running Batch programs keep IMS databases offline
My knowledge of IMS is minimal but I seem to recall there being two flavors of IMS batch processing IMS DL/I was usually offline processing where files could not be accessed concurrently by the onlines IMS BMP (Batch Message Processing) was where the batch jobs accessed the data bases through message processing regions similar to online transaction processors so they could run concurrent with online. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 05/06/2008 04:09:35 PM: It's been a decade or two, but concurrent batch/online was a base feature of IMS. Problems included the difficulty in backing out the effects of a malfunctioning batch program. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Anne Lynn Wheeler Sent: Monday, May 05, 2008 11:29 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: Long running Batch programs keep IMS databases offline The following message is a courtesy copy of an article that has been posted to bit.listserv.ibm-main as well. F [EMAIL PROTECTED] writes: We have IMS 9 on z/OS and I am fairly new to the platform and have a vested interest in fixing it. Every night we have batch programs that run which in return keeps our databases offline for a long time and as a result our applications are not available for processing. I want to know why batch programs and databases cannot both be online at the same time ? If the batch programs read the databases, then why are they offline ? Anyways, what are some ways of ensuring that batch jobs and databases can both run and be online at the same time ? some recent discussions about overnight batch window ... which requires exclusive access to all the information ... as opposed to online. http://www.garlic.com/~lynn/2008b.html#74 Too much change opens up financial fault lines http://www.garlic.com/~lynn/2008c.html#92 CPU time differences for the same job http://www.garlic.com/~lynn/2008d.html#30 Toyota Sales for 2007 May Surpass GM http://www.garlic.com/~lynn/2008d.html#31 Toyota Sales for 2007 May Surpass GM http://www.garlic.com/~lynn/2008d.html#73 Price of CPU seconds http://www.garlic.com/~lynn/2008d.html#87 Berkeley researcher describes parallel path http://www.garlic.com/~lynn/2008d.html#89 Berkeley researcher describes parallel path http://www.garlic.com/~lynn/2008g.html#55 performance of hardware dynamic scheduling http://www.garlic.com/~lynn/2008h.html#50 Microsoft versus Digital Equipment Corporation there were some number of efforts in the 90s (billions of dollars) that looked at business process re-engineering to leverage killer micros and distributed object-oriented technology to implement straight-through processing (eliminating the overnight batch window). It turns out that many of these had grandious failures when nobody bothered to do any speedsfeeds until very late in the effort ... frequently belatedly discovering that the distributed object-oriented technology had a factor of 100 times increase in overhead (compared to the typical Cobol batch implementation), totally obliterating any hopes of throughput improvements. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Identify matching records
Just remember that using ALLDUPS to do this assumes there aren't already dupes in either of the files. I usually run the inputs through ICETOOL to drop dupes first and then use those files as input to ALLDUPS. //STEP1EXEC PGM=ICETOOL //TOOLMSG DD SYSOUT=* //DFSMSGDD SYSOUT=* //INDD DISP=SHR,DSN=INPUT1 //OUT DD DISP=(,CATLG),DSN=INPUT1.NODUPES, // SPACE=(CYL,(100,100),RLSE,UNIT=SYSDA,DCB=*.IN //TOOLINDD * SELECT FROM(IN) TO(OUT) ON(01,10,CH) FIRST /* //* The use INPUT1.NODUPES and INPUT2 as input to the example below. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 04/25/2008 03:05:54 PM: Thanks, I did go looking for examples in the DFHSORT web site but couldn't locate them. What is this link? Thanks -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Tony B. Sent: Friday, April 25, 2008 12:04 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: Identify matching records From various examples in the DF/SORT web site. //STEP1EXEC PGM=ICETOOL //TOOLMSG DD SYSOUT=* //DFSMSGDD SYSOUT=* //I DD DISP=SHR,DSN=INPUT1 // DD DISP=SHR,DSN=INPUT2 //MATCH DD SYSOUT=* //NONMATCH DD SYSOUT=* //TOOLINDD * SELECT FROM(I) TO(MATCH) ON(01,10,CH) ALLDUPS DISCARD(NONMATCH) matching on columns 1-10. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of George, William (DHCS-ITSD) Sent: Friday, April 25, 2008 1:06 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Identify matching records We have an IMMEDIATE need to identify records in two files that match. One file has several million records while the other has about ½ million. We need to locate all records in the smaller file that are also in the larger file ASAP. I pose this to the group just in case someone can identify a means via SORT, SUPERC or some other means faster than I can research to find an existing method or building one. Thank You for any insights. Bill -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html No virus found in this incoming message. Checked by AVG. Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date: 4/25/2008 7:42 AM No virus found in this outgoing message. Checked by AVG. Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date: 4/25/2008 7:42 AM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html No virus found in this outgoing message. Checked by AVG. Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date: 4/25/2008 7:42 AM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMS allocation in Cylinders using AMS
I'd also ask if when you see the dataset using 3 times what the user requested, are you looking immediately after it's been defined or after data has been loaded into it? It might be extending once it's populated. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 04/16/2008 01:52:52 PM: Gilbert, What is your device geomtery on your CDS BASE DISPLAY? For 3390 it should be 56664 bytes per track and 15 tracks per cyl. If it were set to 3380, I would expect your allocations to be approx 20% smaller than requested. Where in the manual did it tell you not to use Cyls? I've been using Cyls or Tracks for years with no problem. 43 cyls for 30 MB is about right. Might depend on your Cisize. From: Gilbert CardenasSent: Wed 4/16/2008 1:28 PM To: IBM-MAIN@BAMA.UA.EDU Subject: SMS allocation in Cylinders using AMS Hello everyone, I have an unknown that I have not been able to figure out or at least I don't know where to look. I have a user who is trying to define a vsam file using IDCAMS as follows: DEFINE CLUSTER - ( NAME(BLAH.BLAH.BLAH) - LINEAR - REUSE - CYL(30) - SHAREOPTIONS(3 3) ) - DATA - ( NAME(BLAH.BLAH.BLAH.DATA) - ) - CATALOG(BLAH) The problem is, when I look at the dataset (which is SMS managed) it usually ends up being around 3 times larger than what the user requested. Of course the first place I looked was in the dataclas that this dataset gets assigned and the only thing that the dataclas specifies is a volume count of 1. There are no overrides anywhere else for primary or secondary space. Looking at the Access Method Services documentation, it states that you should not use the TRACKS or CYLINDERS parameters. If you use them for an SMS-managed data set, space is allocated on the volumes selected by SMS in units equivalent to the device default geometry. Could the reason that this dataset is being allocated larger than requested have to do with the fact that the sms volumes are mod 9 volumes? Btw, I tried allocating the dataset in Megabytes (30) and I got 43 cyl with 1 extent. I don't know what the conversion of megabytes to cylinders is but this still doesn't seem right. Regards, Gil. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: how fast can I sort on mainframe (using DFSORT)?
Chris, Since I joined the team as the performance lead a couple years ago, Frank now defers these types of questions to me. Everything you (and all the others that responded) have said are good points about DFSORT performance. But the multitude of recommendations just reiterates the point in my original response that there are so many factors, you could spend weeks tuning things that don't improve anything. Just as I preached during my 15+ years of batch tuning, I still say that before you start tuning anything, you have to determine where you are spending time. Then you start your tuning there. I always prefer to get some data (sysouts, SMF, etc.) before I start making recommendations. That is why I asked Pawel to send us a few sample sysouts as a start. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 03/10/2008 12:48:05 AM: Pawel, I have been out of the sorting business for a while, but the target you can go for is what I call 2T, where T is the time to read in the data. The 2T comes from the fact that you have to read it in and then write it out. Writing to sort work files should not be the problem, hopefully the sort is writing to several of them in parallel during the sort phase and reading from them in parallel during the merge phase. EXCP's are an accountant's measurement, not a tuner's measurement. Look at your SMF data and see what the channel connect times are for SORTIN and SORTOUT. If the elapsed time is significantly greater than those two added together then you have to start looking for the bottleneck. My first reaction to your 17M/second was a question. Is that data sorted divided by total time? I assume it is. If so then you are getting about 34M/second on input and output, and that is not a bad number. Tape and DASD inputs and output will give you different numbers. There tends to be more interference with tape files. Fewer channels, no PAV. Back when DASD was real 3390, 4M/second was the target, so you are doing pretty good. I couch that with the fact that I have not run throughput tests on the various DASD subsystems available today. 34M may be great or it may be just average. I don't know. The best thing you can do is give each sort enough memory. You will need to ask Frank Yeager what the target amounts of memory are for a 10G sort. My guess is about 17M to 24M. A 100G sort might need 50M to 75M. A 1G sort might need 5M to 7M. What you are attempting to do is make it so there is no need to do an intermediate merge, or in the case of blockset sorting, that the final merge remains efficient. I would not attempt to get a 10G sort to run as a turnaround or incore sort. It will put a large strain on RSM/ASM and not buy you that much. I hope Frank weighs in on this. Chris Blaicher BMC Software, Inc. These are my own personal comments and do not reflect the views of my employer. Please do your own evaluation of any statements or suggestions made. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Pawel Leszczynski Sent: Sunday, March 09, 2008 4:06 PM To: IBM-MAIN@BAMA.UA.EDU Subject: how fast can I sort on mainframe (using DFSORT)? Hello everybody, I realize subject is VRY broad and my question VRY general, but... Recently in our shop we are reviewing our whole batch processing. Most of the time of EOD processing is consumed by sort of many big sequential files. (One such file has approximately order of 10GB, 10mln records) I listed few tens of such batch jobs (the longest-lasting ones) and computed mean sorting rate. It appeared to be about 1GB/min ~ 17MB/sec I suppose it's very poor result(???). Can you tell me how much I can improve this? These batch jobs are little CPU-consuming (~10% of one CPU), I suppose that major concern is to: -limit EXCPS (1) -increase throughput rate from DASD to central storage (2) I realize that sorting whole file in central storage (hiperspace sorting) would eliminate need to use work files and EXCPS to them. How much central storage is needed to handle in-storage sorting for let's say 10 GB file??? How much can I improve (2)? Can you tell me what is mean sorting rate in your installation? Before starting I would just like to know if I can achieve substantial improvement. TIA, Pawel Leszczynski PKO BP SA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions,
Re: how fast can I sort on mainframe (using DFSORT)?
Pawel, It's really hard to say what is a good sorting rate. There are so many factors that can impact the performance of the sort. File characteristics, i/o contention, storage resources, etc. I have seen customer sorts with much higher throughput rates than you and I've seen some with worse. The DFSORT team is alwasy willing to assist with tuning. If you can send a note to our hotline ([EMAIL PROTECTED]) we can assist you. If possible, add //SORTDIAG DD DUMMY to some of the long running sort steps and send us the sysout. Be sure to send the entire sysout and not just the DFSORT messages. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 03/09/2008 05:05:33 PM: Hello everybody, I realize subject is VRY broad and my question VRY general, but... Recently in our shop we are reviewing our whole batch processing. Most of the time of EOD processing is consumed by sort of many big sequential files. (One such file has approximately order of 10GB, 10mln records) I listed few tens of such batch jobs (the longest-lasting ones) and computed mean sorting rate. It appeared to be about 1GB/min ~ 17MB/sec I suppose it's very poor result(???). Can you tell me how much I can improve this? These batch jobs are little CPU-consuming (~10% of one CPU), I suppose that major concern is to: -limit EXCPS (1) -increase throughput rate from DASD to central storage (2) I realize that sorting whole file in central storage (hiperspace sorting) would eliminate need to use work files and EXCPS to them. How much central storage is needed to handle in-storage sorting for let's say 10 GB file??? How much can I improve (2)? Can you tell me what is mean sorting rate in your installation? Before starting I would just like to know if I can achieve substantial improvement. TIA, Pawel Leszczynski PKO BP SA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT anomaly?
Herbie, Send the sysout from your job directly to me ([EMAIL PROTECTED]) and I'll take a look. We may need you to open an ETR on this but let me take a look first. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 01/22/2008 07:23:16 AM: Kees, What bugs me is the following... Whether we have 90,000 of 160,000 records, the final number remains Display from COBOL PGM... T735 WRITTEN = 050634 Display from DFSORT...RECORDS - IN: 45900, OUT: 45900 Display from COBOL PGM... T735 WRITTEN = 052687 Display from DFSORT...ICE054I 0 RECORDS - IN: 45900, OUT: 45900 Regards Herbie -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Vernooy, C.P. - SPLXM Sent: 22 Januarie 2008 12:16 nm To: IBM-MAIN@BAMA.UA.EDU Subject: Re: DFSORT anomaly? Vernooy, C.P. - SPLXM [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] t... Van Dalsen, Herbie [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] ... Kees, That is actually that happens... Step1 creates the file, Step2 Makes a backup, using IDCAMS, and the backup does have 16000 records, and step3 does the sort... very weird... //DEL005 EXEC PGM=IEFBR14 //DD3 DD DSN=HUBRECHT.SORTINOUT, // DISP=(MOD,DELETE,DELETE), // UNIT=SYSDA,SPACE=(TRK,0) //* //NCPO511 EXEC PGM=NCPO511 //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //IPMOUT DD DSN=HUBRECHT.SORTINOUT, // DISP=(,CATLG,DELETE), // UNIT=SYSDA,SPACE=(CYL,(15,5)), // DCB=(RECFM=VB,LRECL=4504,BLKSIZE=0) //* //IDCAM020 EXEC PGM=IDCAMS //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FILEIN DD DSN=HUBRECHT.SORTINOUT,DISP=SHR //FILEOUT DD DSN=HUBRECHT.SORTINOUT.BACKUP(+1), // DISP=(,CATLG,DELETE), // UNIT=SYSDA, // SPACE=(CYL,(15,5),RLSE), // DCB=(RECFM=VB,LRECL=4504,BLKSIZE=0) //SYSINDD * REPRO INFILE(FILEIN) OUTFILE(FILEOUT) - REPLACE //* //SORT01 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SORTIN DD DSN=HUBRECHT.SORTINOUT,DISP=SHR //SORTOUT DD DSN=HUBRECHT.SORTINOUT,DISP=SHR //SYSINDD * OPTION COPY INREC IFTHEN=(WHEN=(5,2,ZD,EQ,1,AND,1426,2,CH,EQ,C'48'), OVERLAY=(132:C'5',134:C'S',888:C'CT6')), IFTHEN=(WHEN=(5,2,ZD,EQ,1,AND,150,4,ZD,EQ,6010), OVERLAY=(54:+12,TO=ZD,LENGTH=2)) Regards Herbie I really wonder if it possible to make Sort to this, with sortin and sortout being the same dataset. Make sortout a different dataset and see what happens. And/or check the manual for the validity of this sortstep. Kees. Or wait a few hours until the DFSort support team has woken up and read their mail. Kees. ** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 ** -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html Elavon Financial Services Limited Registered in Ireland: Number 418442 Registered Office: Block E, 1st Floor, Cherrywood Business Park, Loughlinstown, Co. Dublin, Ireland Directors: Robert Abele (USA), John Collins, Terrance Dolan (USA), Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson Elavon Financial Services Limited, trading as Elavon, is regulated by the Financial Regulator -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL
Re: DFSORT anomaly?
I haven't seen any job output yet but I wanted to share this information from DFSORT Application Programming Guide *** For a copy application, the SORTIN data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results. *** Frank probably knows more about this but it makes sense to me why we could have problems for COPY. When we do a SORT, all of the records from SORTIN need to be read before we can write anything to SORTOUT. This is because that last record read could potentially be the first record written. However, when we do a COPY, we can start writing to SORTOUT immediately. I would just try changing your sort step to write a new dataset and see what the results are. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 01/22/2008 08:36:49 AM: What confused me though was why could IDCAMS copy the whole file to a backup? Regards Herbie -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Van Dalsen, Herbie Sent: 22 Januarie 2008 01:20 nm To: IBM-MAIN@BAMA.UA.EDU Subject: Re: DFSORT anomaly? That was my initial thinking, but I did not want to cloud anyone else's thinking... -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Shedlock, George Sent: 22 Januarie 2008 01:05 nm To: IBM-MAIN@BAMA.UA.EDU Subject: Re: DFSORT anomaly? Is it possible that the COBOL program is not closing the file and therefore any additional extents are not reflected in the VTOC / Catalog entry? George Shedlock Jr AEGON Information Technology AEGON USA 502-560-3541 -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Van Dalsen, Herbie Sent: Tuesday, January 22, 2008 5:59 AM To: IBM-MAIN@BAMA.UA.EDU Subject: DFSORT anomaly? Hi all, WE are running z/OS 1.6 with no plans to upgrade because we aren't going to fork out any more for Tivoli 8.3. We also have Enterprise COBOL. This is the problem... The file was allocated Cyl,(15,5) and 16 records written to it by the COBOL program, it thus expanded to 3 volumes and lots of extents... In the same job, this file was sorted, but only 45900 records were sorted, into itself, and the rest of the records were lost. We have tried to re-create the problem by creating the multivolume file with IEBGENER... unsuccessfully... My thinking is this, and I have not found a MVS APAR stating that this might be the case, but here goes... Because of some problem / limitation in the COBOL / MVS File close routines... All three extents of the file are not known to DFSORT 3 steps lower and only at the actual end of the job, but how do I prove this. This occurred 3 times now, until we changed the allocation to Cyl,(200,50) and end up with the file on 1 volume, problem sorted? Any ideas, Any help would be greatly appreciated. Regards Herbie Elavon Financial Services Limited Registered in Ireland: Number 418442 Registered Office: Block E, 1st Floor, Cherrywood Business Park, Loughlinstown, Co. Dublin, Ireland Directors: Robert Abele (USA), John Collins, Terrance Dolan (USA), Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson Elavon Financial Services Limited, trading as Elavon, is regulated by the Financial Regulator -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html Elavon Financial Services Limited Registered in Ireland: Number 418442 Registered Office: Block E, 1st Floor, Cherrywood Business Park, Loughlinstown, Co. Dublin, Ireland Directors: Robert Abele (USA), John Collins, Terrance Dolan (USA), Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson Elavon Financial Services Limited, trading as Elavon, is regulated by the Financial Regulator -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html Elavon Financial Services Limited Registered in Ireland: Number 418442 Registered Office: Block E, 1st Floor, Cherrywood Business Park, Loughlinstown, Co.
Re: DFSORT anomaly?
I would change the dataset names instead of changing the copy to a sort. Today, probably not much difference but if the file size grows over time, the copy is going to be faster and require less resources than a sort. I'd change the following Add a DD5 to your initial DEL005 step with a new data set named something like PECT.TBC.SE.ALC.TEMP.IMPOUT Change NCP0511 step to create PECT.TBC.SE.ALC.TEMP.IMPOUT Change IDCAM020 step's FILEIN to PECT.TBC.SE.ALC.TEMP.IMPOUT Change SORT01 step's SORTIN to PECT.TBC.SE.ALC.TEMP.IMPOUT You could also add another BR14 step right after the SORT01 step to delete the TEMP.IMPOUT if you want to avoid that extra copy occupying space. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 01/22/2008 11:04:26 AM: Thanks to everyone for replying to this. I am definitely going to get the apps guys to change the JCL. I just haven't whether it would be to change it to a sort, or create a second dataset. Will read the recommendations between the lines of the DFSORT manual... Regards Herbie -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Frank Yaeger Sent: 22 Januarie 2008 03:50 nm To: IBM-MAIN@BAMA.UA.EDU Subject: Re: DFSORT anomaly? David Betten wrote on 01/22/2008 06:53:52 AM: I haven't seen any job output yet but I wanted to share this information from DFSORT Application Programming Guide *** For a copy application, the SORTIN data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results. *** Frank probably knows more about this but it makes sense to me why we could have problems for COPY. When we do a SORT, all of the records from SORTIN need to be read before we can write anything to SORTOUT. This is because that last record read could potentially be the first record written. However, when we do a COPY, we can start writing to SORTOUT immediately. I would just try changing your sort step to write a new dataset and see what the results are. Dave has it right. I just got here and was going to write a similar post. You SHOULD NOT use the same data set for SORTIN and SORTOUT for a COPY for the reason Dave states that we are doing parallel read/write. Depending on how the data is arranged, you may get away with it, but you're taking a risk. We can't guarantee that you won't lose records. So just don't do it. You can use the same data set for SORTIN and SORTOUT for a SORT because we don't do parallel read/write. This is commonly known as a suicide sort because the input data set is overwritten so if something goes wrong (e.g. an I/O error) you don't have the input data set anymore. So you would want to be sure you have a backup for the input data set if you do a suicide sort. Frank Yaeger - DFSORT Development Team (IBM) - [EMAIL PROTECTED] Specialties: PARSE, JFY, SQZ, ICETOOL, IFTHEN, OVERLAY, Symbols, Migration = DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html Elavon Financial Services Limited Registered in Ireland: Number 418442 Registered Office: Block E, 1st Floor, Cherrywood Business Park, Loughlinstown, Co. Dublin, Ireland Directors: Robert Abele (USA), John Collins, Terrance Dolan (USA), Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson Elavon Financial Services Limited, trading as Elavon, is regulated by the Financial Regulator -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT question
There can be a slight decrease in performance when you increase the number of sortworks. Going from say 32 to 48 or 64 probably won't have a major impact. However, jumping to something like 128 or 255, you will see a more noticable impact. If you want to send me the sysout from the sort, I can take a look and see if there's any additional tuning we could do to improve performance and/or decrease the sortwork requirement. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 01/14/2008 09:30:08 AM: This just came up. Which is better: to allocate more, smaller, SORTWKnn DD statements, or fewer, larger, SORTWKnn DD statements, or does it not really matter. By better, I mean: Which will result in a shorter run time?. The question came up due to a huge sort this weekend. DFSORT wanted to allocate the SORTWKnn space 3390-3 volume. We only have 3390-3 volumes. So the question is, do I update defaults to have more than 32 volumes or do we consider creating 3390-9 volumes for SORTWKnn? This is IBM's DFSORT on z/OS 1.8. Yes, I am aware of Syncsort's MAXSORT capability. No, it will not be considered. End of discussion on that point, please. -- John McKown Senior Systems Programmer HealthMarkets Keeping the Promise of Affordable Coverage Administrative Services Group Information Technology The information contained in this e-mail message may be privileged and/or confidential. It is for intended addressee(s) only. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited and could, in certain circumstances, be a criminal offense. If you have received this e-mail in error, please notify the sender by reply and delete this message without copying or disclosing it. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Extracting SMF Records to find the userid who deleted a dataset
You can try this //SORT EXEC PGM=SORT,REGION=5000K //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //SORTIN DD DISP=SHR,DSN=inputsmf //SORTOUT DD DISP=(NEW,PASS),DSN=TEMP, // SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG //SYSINDD * SORT FIELDS=COPY INCLUDE COND=(6,1,BI,EQ,X'11',, 45,44,CH,EQ, C'deleted.dsn ') OPTION VLSHRT /* //PRINTEXEC PGM=IDCAMS //IN1 DD DISP=(OLD,PASS),DSN=TEMP //SYSPRINT DD SYSOUT=* //SYSINDD * PRINT INFILE(IN1) DUMP /* Type 17 is pretty easy becaue the dsn is part of the header so you know the offset in each record. Some other smf record types are not so simple because you need to use the triplet for the offset to the section that had the data. Also, I used a length of 44 for the dsn field. If your dsn is less tan 44, you can shorten the length and get rid of the trailing blanks. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 12/13/2007 11:51:04 AM: On one of our account, a production dataset is deleted. We Greetings, On one of our account, a production dataset is deleted. We have to find the userid of the person who deleted that specific dataset. I know that we can extract the SMF records with IFASMFDP and Sort the record type 17 for it. However, I am not sure what would be the sort JCL and sort control statements for it. Could you please guide me or provide a sample jcl which could extract the SMF dataset and find the userid. Any help is greatly appreciated. Thanks Vijay 5, 50, 500, 5000 - Store N number of mails in your inbox. Go to http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Extracting SMF Records to find the userid who deleted a dataset
Frank Yaeger just pointed out to me that the trailing blanks are not necessary since DFSORT will pad the constant on the right with blanks to the length of the field. So you can just code it like this //SORT EXEC PGM=SORT,REGION=5000K //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //SORTIN DD DISP=SHR,DSN=inputsmf //SORTOUT DD DISP=(NEW,PASS),DSN=TEMP, // SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG //SYSINDD * SORT FIELDS=COPY INCLUDE COND=(6,1,BI,EQ,X'11',, 45,44,CH,EQ, C'deleted.dsn') OPTION VLSHRT /* //PRINTEXEC PGM=IDCAMS //IN1 DD DISP=(OLD,PASS),DSN=TEMP //SYSPRINT DD SYSOUT=* //SYSINDD * PRINT INFILE(IN1) DUMP /* Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 12/13/2007 12:45:18 PM: You can try this //SORT EXEC PGM=SORT,REGION=5000K //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //SORTIN DD DISP=SHR,DSN=inputsmf //SORTOUT DD DISP=(NEW,PASS),DSN=TEMP, // SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG //SYSINDD * SORT FIELDS=COPY INCLUDE COND=(6,1,BI,EQ,X'11',, 45,44,CH,EQ, C'deleted.dsn ') OPTION VLSHRT /* //PRINTEXEC PGM=IDCAMS //IN1 DD DISP=(OLD,PASS),DSN=TEMP //SYSPRINT DD SYSOUT=* //SYSINDD * PRINT INFILE(IN1) DUMP /* Type 17 is pretty easy becaue the dsn is part of the header so you know the offset in each record. Some other smf record types are not so simple because you need to use the triplet for the offset to the section that had the data. Also, I used a length of 44 for the dsn field. If your dsn is less tan 44, you can shorten the length and get rid of the trailing blanks. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 12/13/2007 11:51:04 AM: On one of our account, a production dataset is deleted. We Greetings, On one of our account, a production dataset is deleted. We have to find the userid of the person who deleted that specific dataset. I know that we can extract the SMF records with IFASMFDP and Sort the record type 17 for it. However, I am not sure what would be the sort JCL and sort control statements for it. Could you please guide me or provide a sample jcl which could extract the SMF dataset and find the userid. Any help is greatly appreciated. Thanks Vijay 5, 50, 500, 5000 - Store N number of mails in your inbox. Go to http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: KSDS CA split, no CI split
Try increasing your index CI size. Sometimes when the keys are very unique you get a situation where the index CI can't hold enough pointers for an entire data CA. That might be causing the CA splits. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 12/02/2007 06:11:44 PM: The following scenario: KSDS defined with RECSZ(100 100), FSPC(15 15), CISZ(4096). Records loaded, with evenly distributed key values (i.e. start 1, increment 10). 2 records. Then I inserted 500 evenly distributed records (start=5, increment=100). Listcat shows no CI splits and 3 CA splits. Q: how it is possible to have CA split without CI split ? BTW: I increased CA freespace to 30% without effect. Increasing CI freespace finally gave me 0 splits at all. -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sąd Rejonowy dla m. st. Warszawy XII Wydział Gospodarczy Krajowego Rejestru Sądowego, nr rejestru przedsiębiorców KRS 025237 NIP: 526-021-50-88 Według stanu na dzień 01.01.2007 r. kapitał zakładowy BRE Banku SA (w całości opłacony) wynosi 118.064.140 zł. W związku z realizacją warunkowego podwyższenia kapitału zakładowego, na podstawie uchwał XVI WZ z dnia 21.05.2003 r., kapitał zakładowy BRE Banku SA może ulec podwyższeniu do kwoty 118.760.528 zł. Akcje w podwyższonym kapitale zakładowym będą w całości opłacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: How do parallel DFSORTs communicate with each other ?
Good answer Martin. The only thing I'd add is that we have an info APAR, II13495, that provides guidance for setting the DFSORT defaults related to storage. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 11/13/2007 03:01:07 AM: Tobias asked: But i now want to know, where does DFSORT get the information how much storage is allocated to other DFSORT instances. Ahead of Dave Betten (of DFSORT Development) answering here's a reply that I hope will be useful to you... Individual DFSORT instances DON'T communicate with each other. But they do ascertain the state of available memory using the STGTEST SYSEVENT system call. Since 1997 - when we introduced Dynamic Hipersorting with Release 13 - these checks for available memory happen at various stages during the SORTIN phase (while the work files are being built). This repetitive checking regime reduces the risk of two sorts grabbing the same piece of memory. Note: By using STGTEST SYSEVENT DFSORT is actually conditioning its usage not just relative to other DFSORT invocations but relative to other address spaces and system conditions in general. In short, even in the same address space (which is one of the DB2 Version 8 Utilities scenarios - with Parallel Index Build etc) DFSORT invocations don't communicate with each other. But then you don't need them to. :-) Dave's a great friend and I'm sure he'll give you a better answer. But he won't give it to you for another 4 or 5 hours. :-) Cheers, Martin Martin Packer Performance Consultant IBM United Kingdom Ltd +44-20-8832-5167 +44-7802-245-584 [EMAIL PROTECTED] Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Performance comparison: Hiperbatch, BLSR, SMB?
Also, remember that for Hiperbatch only the Data component is kept in the DLF object. So you would still have to deal with I/Os to the index. You can provide enough buffers to keep the index set in storage but you don't get lookaside on the sequence set. With BLSR, you can avoid the seqence set I/Os too. So you have to see which is going to provide the greater benefit, eliminating all of the data I/Os with Hiperbatch, or eliminating more index I/Os (and hopefully some data I/Os) with BLSR. Which provides greater benefit is going to depend on the reference patterns of the jobs accessing the file (that's a fancy way of saying YMMV!). :-) Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 11/12/2007 07:44:59 AM: Alex Tough asked: If you have all or most of the cluster allocated to a DLF data object, why would BLSR be faster ? It might depend on what you mean by faster... Note that nowadays a DLF (or any) hiperspace would be backed by central but would still require a more distant data movement. And, in principle, VSAM LSR (whether via BLSR or not) ought to be in the job's own address space. So, in CPU terms I'd speculate that BLSR might cost less CPU. That probably wouldn't turn into elapsed time differences that were measurable - but it just MIGHT. Another thing to remember, though, is that Hiperbatch keeps the memory pro bono publico whereas BLSR doesn't. The affordability of Hiperbatch may be much greater for CONCURRENT jobs. In the same vein with multiple BLSR jobs each job would have to populate the buffer pool itself. So initially you might get a lot of misses. All a long-winded way of saying YMMV. :-) Cheers, Martin Martin Packer Performance Consultant IBM United Kingdom Ltd +44-20-8832-5167 +44-7802-245-584 [EMAIL PROTECTED] Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Performance comparison: Hiperbatch, BLSR, SMB?
Remember there are two types of DLF objects (Hiperbatch Retain and non-Retain) Non-Retain is the one that gets deleted when the open count for the dataset is 0. This is the one that was intended for sequential use and you didn't need a DLR object as large as the dataset. Just one large enough that concurrent sequential readers would benefit. Reader 1 reads from disk and places copy in Hiperbatch, readers 2, 3, 4, etc. read the Hiperbatch copy. If the DLF object can hold 100 records than you just hope the other readers have all gotten to record 1 before the first one gets to record 101. Sort of a moving window. Retain is the one that stays there until you explicility delete it. This is good for both sequential and random, especially if it's large enough to fit the entire dataset. You load the whole thing into memory once and then everyone reads the Hiperbatch copy in memory. Just thought I'd pass on that distinction since it's important when talking about whether sequential or random can benefit. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 11/09/2007 04:07:56 PM: -Original Message- From: Gerhard Adam [mailto:[EMAIL PROTECTED] Sent: Friday, November 09, 2007 1:57 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: Performance comparison: Hiperbatch, BLSR, SMB? Unfortuanately I haven't looked up this stuff in a long time, so I might be wrong. But IIRC, Hiperbatch is intended for sequential access and is counter-productive for random files. Since it uses a Most Recently Used algorithm (instead of LRU), the intent was to ensure that the most recent access to a record was the most eligible for getting discarded from memory (since this represented the last reader of the data). Well, I don't know about your memory, but the latest version does indeed support random access, AFTER the DLF has been loaded sequentially. Don't know what algorithm it uses, but my SMF gurus tell me they can prove it is being used, and in fact the batch jobs run longer if anything happens to Hiperbatch, so I think it's working. Our access is almost all random. The whole point was to avoid having records discarded because of age just ahead of someone that was reading the file sequentially. Also, another point was that the I/O counts were unaffected since the application was unaware that it was using Hiperbatch, so that information is largely irrelevant. I didn't know that, but it makes sense. Thanks. Anyway ... here's hoping my memory isn't completely gone Not completely. :) Peter This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Each CPU usage
I think the debate of whether this story is plausible or not is better taken offline. Or you can just agree to disagree. I think the important point that everyone would agree on is that when you are reaching maximum CPU capacity, it's always a good idea to gather data and understand what's driving that CPU before you upgrade. You may find tuning opportunities to reduce the cpu requirements, or you may just get a better understanding of what workloads are driving that usage and what kind of latend demand you have. 100% CPU might not be so bad if your critical work is getting what it needs and discretionary work is taking the rest (I know the decision on what's discretionary can be a political battle!). CICS performance statistics (SMF 110) have all kinds of great information to help you understand the characteristics of your transaction workload including CPU usage. I'm no CICS expert but I've seen that data put to great use by those who know how to use it. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 08/29/2007 08:55:47 AM: It was a custom application written for a very large customer. Someone else mentioned that it seemed an improbable scenario. Be that as it may, it happened. The problem isn't simply with the scenario of acquiring a lock, but rather than there was an apparent upgrade of 250% (regarding the number of engines) and apparently NOBODY bothered to gather data to determine the basis for such a change. Secondly, your assertion that the processor was at 100% utilization, suggests that the four engines were occupied in spin loops, which also suggests that no other work was capable of running (which is another implausible scenario). Since the conclusion was that they didn't even require a 4-way after the problem was solved, it is clear that the utilization of the processor was driven by the spin loops which leads one to conclude that there could have been no other running work on the system beyond four transactions. As I indicated, this stretches the bounds of even anectodal evidence and would require a much more detailed explanation to approach plausibility. Adam -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: What's the new support for large files called?
Check out this manual for a good description of large format datasets. z/OS DFSMS: Using the New Functions Document Number SC26-7473-02 Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 08/03/2007 01:46:12 PM: -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Lindy Mayfield Sent: Friday, August 03, 2007 12:40 PM To: IBM-MAIN@BAMA.UA.EDU Subject: What's the new support for large files called? When you forget the terminology it's impossible to Google or anything. Sorry about the stupid question, but: What is this new support for large datasets in, I think z/OS 1.7 or 1.8 called? I wanted to Google to find the size differences. I've Googled for 10 minutes and without the proper keywords it isn't easy. (-: Thanks! Lindy Do you LBI? That's for LRECL 32K. Or Large Dataset (extended format) for datasets with 65K tracks on a single volume. -- John McKown Senior Systems Programmer HealthMarkets Keeping the Promise of Affordable Coverage Administrative Services Group Information Technology The information contained in this e-mail message may be privileged and/or confidential. It is for intended addressee(s) only. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited and could, in certain circumstances, be a criminal offense. If you have received this e-mail in error, please notify the sender by reply and delete this message without copying or disclosing it. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SOC4 in DFSORT with DB2
I don't want to circumvent the PMR process but thought I'd offer some a thought based on this information. Double check the storage class that the sortwork datasets are being assigned to and make sure the default unit count is 1. Sortwork datasets cannot be multi-volume so there's no point in having a default greater than one. If the default volume count is something high, like 59, then you're probably going to exceed the TIOT threshold. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 08/01/2007 11:30:01 AM: George, I should have mentioned that our Abend0C4 was in IEFAB42A, or IEFAB421, depending on what you are looking at. John Confidentiality notice: The information included in this e-mail, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized review, use, disclosure, distribution or similar action is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the original message immediately. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Tergerson, John Sent: Wednesday, August 01, 2007 11:21 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: SOC4 in DFSORT with DB2 George, We got an 0C4, and also a U0083 symptom using DFSORT to do reorgs on a tablespace with a lot of indexes. The DBA had set up sortnum to 160 so they would never have to compute sort work space. When they reduced it to 80, things worked fine. The tablespace with the problem was small, but has lots of indexes. Large tablespaces with fewer indexes could be reorged with larger sortnum. There are two new APARs we caused. Look at OA21854 and darn, I can't get to IBMLink to find the other, if it was even assigned yet. In our case, the large number of sortnum caused a condition that exceeded the allowable size of the TIOT. The allocation APAR is to give the user a better symptom than abend0C4 to tell the user what is wrong. The 0C4 is because the slot allocation was trying to free up was for VIO, which does not have a control block they were looking for. The DB2 APAR is to address an issue of why they were asking for an enormous number of sortwork datasets. I'll post that whenever I can find it. Asking for sortnum of 80 for most reorgs, though, gave us a welcome bypass. Hope that helps. John Confidentiality notice: The information included in this e-mail, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized review, use, disclosure, distribution or similar action is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the original message immediately. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of George Bly Sent: Wednesday, August 01, 2007 8:22 AM To: IBM-MAIN@BAMA.UA.EDU Subject: SOC4 in DFSORT with DB2 HI All We went to DB2V8 this week. Now we are getting S0C4's in DFSORT when using DB2 utilities. I've read the informational APAR and I have it in correctly after my SyncSort libs. I am working with IBM but has anyone else run into this problem. By the way: IBMLINK went down when we first started researching the problem. It was very inconvenient to say the least. Thanks!! George -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT V1R5 - ICE751I and ICE752I
If your system is at z/OS 1.7 or higher, you can exceed 4000 cylinders on a single volume by adding DSNTYPE=LARGE to your sortwork DDs and they will be allocated as large format datasets. If you don't code JCL sortworks and let DFSORT dynamically allocate the required sortwork space ( our recommended method), the sort work datasets are automatically allocated as large format. For your other points, I'll take those under advisement with my team. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 07/01/2007 10:38:55 PM: Hi, Thanks everybody for the feedback. Somebody else was working on the problem today because I actually tried to get some sleep. So I forwarded all these postings to him and Dave , we would sent you the FULL output. Summary: I really would suggest that we make IBM Software more user friendly : a) Why not print a warning message to the User indicating that any SORTWK99 dataset will only use ONE volume b) Why not print a message to the user stating that the Memory allocation is too small. c) Why do we still have a limit in zOS with regards the Primary Cylinder allocation. Apparently it's 4000 Cylinders and I found this out the hard way last night. d) When abending, please tell me more. Currently you print messages that says For the IBM support rep. I thought IBM stopped doing this in the 80's You do print how many records are about to be sorted, so you are half way there ( Don't take me halfway... I want it all and I want it now ) Anton _David Betten wrote: I already answered Anton in another post but I wanted to respond to this and make clear that DFSORT work files cannot span multiple volumes. If you coded UNIT=(SYSDA,10) on a SORTWKnn DD statement, DFSORT would still use only 1 volume for each SORTWK data set. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 07/01/2007 02:10:23 PM: If necessary and the file is SMS managed add a volume parameter to the UNIT parameter, eg. UNIT=(SYSDA,10). It doesn't have to be SMS managed for the volume parameter to work. - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT V1R5 - ICE751I and ICE752I
Anton, Lizette is correct that the diagnostic messages are for use by our L2/L3 teams and that's why they are not documented. The ICE751I and ICE752I messages are informational (note they end with an I) and are issued for both successful and unsuccessful sorts. Usually when a sort fails, there is an ICExxxA message to indicate the cause. If you would like to send me the sysout of your failed sort, I'd be happy to look at it and help determine the cause of your abend. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 07/01/2007 11:27:46 AM: Anton, I am not sure what is your issue. Not all IBM messages are documented because some are for internal diagnostics by the IBM L2/L3 teams (if needed). So I would not expect every message documented because it would be meaningless to me since I do not have access to the IBM Code for DFSORT. It seems to me you are upset over the fact that the job ran for 2 hours before abending. That maybe a valid issue. Since you did not post any specifics (SORT CONTROL CARDS, type of data being sorted or how much data being sorted) it is hard to tell if the sort needs tuning. If you turned the DIAGs on for this job, then you probabaly need to open an ETR with IBM to resolve the B37 issue. Lizette Hi, I abended twice this weekend on SB37-04, after running for TWO hours and when I look at the output of the DFSORT , I see ICE751I and ICE752I messages. The IBM manual says : Explanation: DFSORT diagnostic message for use by IBM software support personnel. Conclusion : That sounds line s'mee but it still does not mean much. Can the IBM DFSORT team not hire a H1B visa person for $8 and hour and document all these possible output messages ? Anton -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT V1R5 - ICE751I and ICE752I
Anton, If you send the entire sysout directly to my email ([EMAIL PROTECTED]) I can probably diagnose your abend easier.But at first glance, I suspect the very limited main storage you are allowing the sort (524288) is causing an intermediate merge and that's when you are running out of sortwork space. But again, I'd need to see the entire sysout to be sure. Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 07/01/2007 12:38:15 PM: Yes... thanks for all the feedback and my initial posting was more out of frustration with DFSORT and SMS. So here comes more detail : The Scene of the crime : Input Dataset contains 4,826,605 records of 2000 bytes each. The Sort Control Cards : ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 03:58 ON SUN JUL SORT FIELDS=(49,6,A,59,8,A,91,6,A,99,8,A,81,2,A, 97,2,A,1947,3,A,55,4,A,1950,2,A),FORMAT=CH ICE201I E RECORD TYPE IS F - DATA STARTS IN POSITION 1 ICE751I 0 C5-K90007 C6-K90007 C7-K9 C8-K90007 E4-K90007 C9-BASE E5-K14794 ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED ICE088I 1 HWLK221A.STEP220 .SORT, INPUT LRECL = 2000, BLKSIZE = 4000, TYPE = ICE092I 0 MAIN STORAGE = (350,524288,524288) ICE156I 0 MAIN STORAGE ABOVE 16MB = (458304,458304) ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y, ICE128I 0 OPTIONS: SIZE=524288,MAXLIM=2097152,MINLIM=524288,EQUALS=Y,LIST=Y,ERET ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=Y,CHALT= ICE130I 0 OPTIONS: RESALL=0,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,S ICE131I 0 OPTIONS: TMAXLIM=5242880,ARESALL=0,ARESINV=0,OVERRGN=0,CINV=Y,CFW=Y,DS ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITC ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMI ICE235I 0 OPTIONS: NULLOUT=RC0 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN ICE750I 0 DC 0 TC 9653212000 CS DSVVV KSZ 45 VSZ 45 Sort Step JCL : XXSORT PROC PROG=ICEMAN, XXWTR1='*', XXCYL=15, XXDEVTYP=TMPDA, XXRGIN=6M XXSORT EXEC PGM=PROG,REGION=RGIN,PARM='SIZE=350' IEFC653I SUBSTITUTION JCL - PGM=ICEMAN,REGION=6M,PARM='SIZE=350' XXSYSOUT DD SYSOUT=WTR1 IEFC653I SUBSTITUTION JCL - SYSOUT=* XXSYSOUX DD SYSOUT=WTR1 IEFC653I SUBSTITUTION JCL - SYSOUT=* XXSYSLMOD DD SPACE=(3600,(20,20,1)),DSN=GOSET, XX UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(3600,(20,20,1)),DSN=GOSET,UNIT=TMPDA XXSYSLIN DD SPACE=(80,(10,10,)),DSN=LOADSET, XX UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(80,(10,10,)),DSN=LOADSET,UNIT=TMPDA XXSYSUT1 DD SPACE=(1000,(60,20)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(1000,(60,20)),UNIT=TMPDA //SORTWK01 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) //SORTWK02 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK02 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SORTWK03 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK03 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SORTWK04 DD SPACE=(CYL,(2000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK04 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SORTWK05 DD SPACE=(CYL,(3000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK05 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SORTWK06 DD SPACE=(CYL,(3000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK06 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SORTWK07 DD SPACE=(CYL,(1000,500)),UNIT=TMPDA, //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB) X/SORTWK07 DD SPACE=(CYL,(CYL,CYL)),UNIT=DEVTYP //* IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA //SYSPRINT DD SYSOUT=* //SORTIN DD DSN=HWL.HWLK228P.ARCHIVE,DISP=OLD //SORTOUT DD DSN=HWL.HWLKSORT.HWLK228P, //DISP=(NEW,PASS,DELETE), //STORCLAS=LRGETAPE,UNIT=3490, //DCB=(HWG.PATTERN.RECL2000,BLKSIZE=0) //SYSINDD * //* The WORK packs available : We have 7 work packs with the following FREE space : WORK00 9,800 Cylinders WORK01 9,861 Cylinders WORK02
Re: DFSORT V1R5 - ICE751I and ICE752I
I already answered Anton in another post but I wanted to respond to this and make clear that DFSORT work files cannot span multiple volumes. If you coded UNIT=(SYSDA,10) on a SORTWKnn DD statement, DFSORT would still use only 1 volume for each SORTWK data set. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 07/01/2007 02:10:23 PM: If necessary and the file is SMS managed add a volume parameter to the UNIT parameter, eg. UNIT=(SYSDA,10). It doesn't have to be SMS managed for the volume parameter to work. - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: RESTART SINGLE JOB STEP
I THINK (not positive) that if you added COND=(0,LE) to the job statement, that would force all the steps after the restart step to flush. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 04/10/2007 12:50:05 PM: RESTART=stepnameor step+proc However, that restarts at the given step but continues to execute the rest of the steps following it. I think she wanted to execute just one step in the middle of the job. -- Bruce A. Black Senior Software Developer for FDR Innovation Data Processing 973-890-7300 personal: [EMAIL PROTECTED] sales info: [EMAIL PROTECTED] tech support: [EMAIL PROTECTED] web: www.innovationdp.fdr.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSort oddity
Neal Eckhardt wrote: I have been testing different values for HIPRMAX to limit the amount of expanded storage it uses. It appears that DFSort uses a Hiperspace OR a Data Space. Specifying HIPRMAX=1% will result in no usage of Data Spaces. Why not? The EXCP counts on SORTIN and SORTOUT are different depending on wether a Hiperspace is used or a Data Space is used (counts below). Why would SORTIN and SORTOUT EXCP counts be different? Neal Dataspace: DEVICE ACTIVITY: (BLOCK COUNT, DDNAME, DEVICE, DEVICE TYPE) 229 SORTIN 1123 3390125 SORTOUT 1228 3390 193 SORTWK04 103E 339 195 SORTWK02 103F 3390191 SORTWK01 1227 3390 184 SORTWK03 1229 339 2 SORTWK05 1228 3390 1 SORTWK06 1228 3390 Hiperspace: DEVICE ACTIVITY: (BLOCK COUNT, DDNAME, DEVICE, DEVICE TYPE) 117 SORTIN 1123 3390299 SORTOUT 1227 3390 346 SORTWK02 103E 3390 332 SORTWK05 103E 3390340 SORTWK03 103F 3390 321 SORTWK01 1228 3390 321 SORTWK04 1229 3390 1 SORTWK06 1227 3390 You are correct that DFSORT will use Hiperspace OR Dataspace. It will not use them together. Note that in DFSORT V1R5 it's Memory Object or Hiperspace or Dataspace. When you specify HIPRMAX=1% and there is no usage of Dataspace, does the sort use Hiperspace instead or neither? The EXCP counts are going to vary between the two methods because of the way DFSORT allocates storage for i/o buffers. DFSORT allocates different control blocks and internal storage areas differently for the two methods. If you would like to investigate this in more detail, you can send me the sysouts from your runs and I can take a look at them. It would be helpful to understand what your objective is in this testing. Generally, we recommend you let DFSORT select the method that's optimal for each sort based on the characteristics of the sort and the available resources on the system. Dave Betten IBM DFSORT Performance Lead [EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: ACS Routine works for JCL but not in ISPF 3.2
Does the data set get assigned to the same storage class when allocated via JCL and ISPF 3.2? Storage class is assigned prior to storage group and your storage group routine could be assigning based on the storage class. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] 1-240-715-4655, tie line 268-1499 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 01/02/2007 02:32:58 PM: -Original Message- From: Vinson Lee [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 02, 2007 1:25 PM To: 'IBM Mainframe Discussion List' Cc: McKown, John Subject: RE: ACS Routine works for JCL but not in ISPF 3.2 I don't see a reference to XMODE in any of our ACS routines. Is there maybe a default? Thanks. Vince XMODE means execution mode. Its value is set depending on the environment during the execution of the ACS routine. However, if you don't reference it in your ACS routine, it does not used for anything. Likely you will need to post your ACS routine, otherwise we are all flying blind. -- John McKown Senior Systems Programmer HealthMarkets Keeping the Promise of Affordable Coverage Administrative Services Group Information Technology The information contained in this e-mail message may be privileged and/or confidential. It is for intended addressee(s) only. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited and could, in certain circumstances, be a criminal offense. If you have received this e-mail in error, please notify the sender by reply and delete this message without copying or disclosing it. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT SORTWK questions
See my answers below: We had a very large DFSort job that abended because of insufficient Sortwk space with message ICE083A, indicating that 4 times 65535 tracks were not enough. I'm guessing your system is not a z/OS 1.7 yet. Just an fyi that for systems at 1.7 or later, DFSORT allocates the sortworks as large format so they can exceed 65535 tracks (providing there are volumes with that much space available). Looking for options to raise the default of 4 Sortwk's, I ran into 2 questions to which I could not find the answer: 1. I cannot find any recommandations in the DFSort manuals about the optimum number of Sortwk's. Spreading more Sortwk's about more devices is in generally advisable, but in modern storage devices less relevant than before. Does DFSort really not care whether it has 5, 50 or 255 Sortwk's and how do I determine the optimum number of Sortwk's? There's not much performance difference when you go up to say 16 or 32. However, you don't want to do something drastic like 255. The reason is that it can cause the sortworks to become fragmented and then the merge of all those strings can take longer. It's difficult to proclaim one specific number as optimal because factors like I/O contention in your environment can influence what performs best. 2. From the manual I understood that the number of Sortwk's that I specify in the DYNALOC parameter of the ICEMAC macro's specified the *maximum* number of Sortwk's DFSort was allowed to take and that it does not force dymanic allocation, nor the number of Sortwk's. Tests we did seem to show that DFSort, when it dynamically allocates Sortwk's, will always allocate the number of Sortwk's in the ICEMAC macro. Is this correct? DFSORT will almost always allocate the number of sortworks you specify in the DYNALOC (installation default) or DYNALLOC (runtime specified). However, if you have JCL sortworks coded but DYNAUTO=IGNWKDD is not in effect, then it will use the JCL sortworks and not perform dynamic allocation. When dynamic allocation is used, the sortwork space required is spread across all the sortworks. So more sortworks does not mean more space, but instead it means more/smaller sortwork datasets. I'm sorry if our manuals are unclear and will see if we can revise to make it more clear. I'll add that to the list of things to consider for future releases. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DFSORT and SS
Frank Yaeger is always the best at answering these types of questions but
he's on vacation this week. So I forwarded your question to Vicky Vezinaw
and she supplied the following answers.
ANSWER to Q1:
When you say:
I used the following syntax:
OMIT COND=(5,32000,SS,EQ,C'SMITH')
OPTION COPY,VLSHRT
Unfortunately it doesn't work.
Did you get a syntax error and are you using DFSORT R14 with the above
statements? If not please be more specific about the problem you are
encountering. If you are getting a syntax error, then you need PTF UQ90053
applied. In that PTF, the maximum length for an SS field used with OUTFIL
INCLUDE and OUTFIL OMIT has been raised to 32752.
ANSWER to Q2:
-
You can run a simple job to get the minimum and maximum record length. I
derived this JCL from an example in the DFSORT Application Programming
Guide (DISPLAY Operator example)
DISPLAY FROM(INV) LIST(RDWLIST1) -
TITLE('No Frills RDW Report') -
ON(NUM) -
ON(VLEN) -
ON(1,4,HEX) -
MINIMUM('Smallest') -
MAXIMUM('Largest')
The INV is the DD for the input data set. RDWLIST1 is the DD for the
output report data set. The rdw's will be printed with the smallest and
largest RDW printed at the end of the report.
ANSWER to Q3:
-
You can use OUTFIL to accomplish this task. I also derived the following
JCL from an example in our DFSORT Application Programming Guide (OUTFIL
description, under the SAVE description)
OUTFIL INCLUDE=(8,6,CH,EQ,C'ACCTNG'),FNAMES=GP1
OUTFIL SAVE,FNAMES=NOTINC
The DD GP1 data set will have the included records. The DD NOTINC data set
will have all of the other records (in other words the records that did not
meet the INCLUDE condition).
I hope this helps you.
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email: [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 08/01/2006
12:41:43 PM:
I have variable length records to filter.
I should omit all records containing given string, i.e. SMITH.
I used the following syntax:
OMIT COND=(5,32000,SS,EQ,C'SMITH')
OPTION COPY,VLSHRT
Unfortunately it doesn't work. When I changed the lenght:
OMIT COND=(5,900,SS,EQ,C'SMITH')
then some records were filtered. I suspect that only records longer than
904 bytes and containing SMITH in first 900 bytes (excluding RDW).
Q: How to fix it, I mean omit all records containing SMITH, regardless
of the length ?
Q2: What is simple method to find out records length, one by one, or
just min and max ?
Q3: How can I split output, put record without the string in one
dataset, and the rest (with the string) in another one ? Should I use
OUTFIL or rather something else ?
Regards
--
Radoslaw Skorupka
Lodz, Poland
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF help needed please
Those don't look like SORT messages but rather something that would come out of the RMF post processor or exits. Does your job have an EXITLIB or STEPLIB that's pointing to downlevel version of the code? Was there a message id at the beginning of those messages like ERB that would indicate who is issuing them. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 06/26/2006 06:35:05 AM: Folks, We recently upgraded to z/OS V1.7 from 1.6 and the JCL that I used under version 1.6 to sort the SMF tape before generating SMF reports no longer works. I get the following SMF message: RECORDS WITH VERSION 717 WERE FOUND IN THE INPUT DATA SET. THE EXPECTED VERSION NUMBER IS 715 RECORDS WITH VERSION NUMBER 7.1.7-74 WERE FOUND IN THE INPUT DATASET. THE EXPECTED VERSION NUMBER IS 7.1.5-71. My old sort parms were: //SYSIN DD * INCLUDE COND=(06,1,BI,GE,X'46',AND,06,1,BI,LE,X'4F') SORT FIELDS=(11,4,CH,A,7,4,CH,A),EQUALS MODS E15=(ERBPPE15,500,,N),E35=(ERBPPE35,500,,N) END /* Does anyone have the new sort parms for version 1.7 that they would be willing to share ?? Or if it is not the sort parms, be able to tell me what is wrong with my JCL? Thanks in advance. Richard W. Evans Technical Specialist Worldspan Worldwide Technology Planning Implementation Department System Services Group z/OS (MVS) Capacity/Performance E-MAIL : [EMAIL PROTECTED] Worldspan e-Pricing ... The next Generation in fare search technology - http://www.epricing.net NOTICE: This e-mail message and all attachments transmitted with it contain confidential information intended solely for the use of the addressee. The addressee agrees to keep the information as confidential. Any improper use of the information, or dissemination or distribution to third parties, is strictly prohibited. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete this message and all copies and backups thereof. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: z/OS DASD Allocation using Esoterics
You may want to look at the DFSMSdfp Storage Administration Reference topic 6.6, SMS Volume Selection for Data Set Allocation. This discusses in detail the criteria used by DFSMS for volume selection. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 06/15/2006 03:17:21 PM: Raymond's point is excellent. It is also possible that there was not adequate space or there was some other constraint at the time SMS made the decisions. It would be interesting to see if you got the exact same results at different periods, repeating the same experiment. I think I'm in trouble here because I just implied that the software can actually THINK! I must need a vacation. Regards, Rob Weiss z/SWITA and z/Series I/T Security and Privacy Consultant IBM Software Group Sales IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 06/15/2006 01:01:40 PM: Raymond, Are all of the SMS managed volumes in ENABLE status? Maybe one of the volumes is in QUIESCE status or even DISABLE or DISABLE, NEW. Regards, John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dcollect
You can download a sample for processing DCOLLECT with DSFSORT from the DFSORT web site. http://www-03.ibm.com/servers/storage/support/software/sort/mvs/srtmdwn.html Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 04/04/2006 03:25:56 AM: There are quite a few DCOLLECT processing examples on the CBT Tape - www.cbttape.org Download File #1 and have a look. Rob Scott Rocket Software, Inc 275 Grove Street Newton, MA 02466 617-614-2305 [EMAIL PROTECTED] http://www.rs.com/portfolio/mxi/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: DB2/DFSORT/MOSIZE/IEFUSI Super Sorting above the bar with DFSORT
I need to comment on a few items here. First, I'm pretty sure that the MEMLIMIT in SMFPRMxx is a default that's used if you don't have MEMLIMIT on your JCL or REGION=0M. I don't believe it overrides it. So if you really want to put a hard limit on the amount of above the bar storage someone can get, you need to put it in the IEFUSI. I know there's a good flow chart on this, I just need to find it again. Second, if DFSORT can't use Memory Object, it can still use your central storage via Hiperspace or Dataspace so you may want to look at your HIPRMAX and DSPSIZE parameters in ICEMAC as well. Third, generally we recommend customers use the shipped defaults of MOSIZE=MAX, HIPRMAX=OPTIMAL and DSPSIZE=MAX. DFSORT is designed to look at available resources and not over commit. But if you don't have a paging subsystem configured to really back a 30GB central, then I agree you need to limit it. You might want to consider leaving MOSIZE=MAX and HIPRMAX=OPTIMAL and setting EXPMAX to a value you are comfortable with. EXPMAX sets the limit of central storage (in 64-bit env) that can be used by ALL Memory Object and Hiperspace sorts in the system. You should also look at EXPOLD which says how much old storage is available. I suspect the paging you saw was because DFSORT went after old pages from long running address spaces like your online subsystems and that caused them to page out. You could do something like leave EXPMAX=MAX and change to EXPOLD=10%. This will allow DFSORT to obtain any available pages but limit page steals from any other address spaces. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 03/21/2006 03:53:04 PM: Hi, We have a few exploiters of storage above the bar and mostly it has been all good but I have seen some interesting things with DFSORT recently. DFSORT using the default MOSIZE=MAX used 12G for an index rebuild called by DB2 utility PGM=DSNUTILB,REGION=0K on an LPAR with 30GB running the AFQ down to 18326 and inducing some paging on a system where I would not have expected to see it. I specify MEMLIMIT(4G) on most of our systems in SMFPRMxx. Currently since we use DFSORT only for DB2 V8 I had not prompted anyone to customize ICEMAC. I recently added an IEFUSI exit to the mix here but don't yet have any code in there to handle MEMLIMIT. I am considering adjusting both with my current thinking to change IEFUSI to honor the specification from SMFPRMxx so only GRS/DB2/reallycoolauthorizedguys will override that and to customize ICEMAC to change MOSIZE=MAX to MOSIZE=10% to prevent any single DFSORT job from getting too giddy when it sees a large AFQ. What are you doing here? Anyone else customize MOSIZE in ICEMAC? Anything else in the ICEMAC defaults to watch out for? We have been a long time SYNCSORT shop so have been taking a wait and see approach with DFSORT tuning. Best Regards, Sam Knutson, GEICO Performance and Availability Management mailto:[EMAIL PROTECTED] (office) 301.986.3574 Before you criticize someone, you should walk a mile in their shoes. That way, when you criticize them, you're a mile away and you have their shoes. ][ This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: CFW and DFSORT
I apologize, my earlier append was not clear. DFSORT does check and if CFW is not active in the controller, then the CFW bit is not turned on. Also, I would agree that there is a performance benefit of using CFW since you eliminate the need to write to the NVS. But I have seen many customers running effectively with CFW=NO. So if you're looking for the high availability of a GDPS environment and Hyperswap, I don't think disabling CFW is going to be a show stopper. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@BAMA.UA.EDU wrote on 02/24/2006 12:16:15 AM: Bruce, Yes I misread that. But with CFW ON in DFSORT and OFF in the Controller I don't think SORT turns on the CFW bit. I think CFW ON in DFSORT causes a check for CFW ON in the controller, and then CFW is set appropriately. David's reply suggests that SORT sets the CFW bit even when the controller has CFW OFF. The IO chaining behaviour suggests otherwise. Ron -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Bruce Black Sent: Friday, 24 February 2006 10:52 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: CFW and DFSORT You're from IBM right? So you are saying that ESS and DS8K ignore CFW and put a copy in the NVS anyway? Then why does DFSORT change its IO chaining behaviour when CFW is on. No, he said that if you disable CFW in the control unit but enable it in DFSORT, the control unit ignores the CFW request and treats it as a regular write. -- Bruce A. Black Senior Software Developer for FDR Innovation Data Processing 973-890-7300 personal: [EMAIL PROTECTED] sales info: [EMAIL PROTECTED] tech support: [EMAIL PROTECTED] web: www.innovationdp.fdr.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: CFW and DFSORT
You should not have a problem with DFSORT if you disable cache fast write in your contoller. Even if the DFSORT default is CFW, the controller will simply ignore the CFW flag and treat it as a normal write. Generally, I have not seen a significant perfomrance impact to DFSORT when CFW is turned off. This is mainly due to the larger NVS sizes and high speed interfaces of today's storage technology. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Mainframe Stress Testing
Loadrunner actually does have a 3270 interface now so it can drive terminal traffic to a mainframe CICS application. I believe it can simulate multiple users using a tn3270 emulator to access the mainframe application. No matter what tool you use, the real work is in developing an accurate workload. You have to make sure you have a realistic transaction mix and you have to get valid data to feed into the scripts. This usually involves data base queries to generate lists of valid acct numbers, etc. Then once you get it all working, you have to keep the data and the scripts current. Have a nice day, Dave Betten DFSORT Development, Performance Lead IBM Corporation email: [EMAIL PROTECTED] DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
