Re: SMF volume - follow-up
On Fri, 22 Jun 2012 11:29:59 -0700, Edward Jaffe edja...@phoenixsoftware.com wrote: On 6/22/2012 7:00 AM, Mark Zelden wrote: Reminds me of a joke in the recording studio when doing a mix and to make everything louder than everything else. :-) Or when on-stage and everyone keeps turning up their volume... :-) That's why my amp goes to 11! :) Art -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
Barbara Nitz wrote: This is just type 100 - 102 records on 2 out of 11 LPARS BTW. For some reason - the word overkill came to mind. Yes, whoever had the bright idea to put DB2 tracing into SMF should be shot. Or place those DB2 subsystems in 'Siberia' (Special WLM class) and grab a small dump (to SYSOUT) of the last 1000 SMF records to see what and where are they. Then contact these owners. Repeat offenders get a modify command to stop. If that does not work, then cancel. We run flatline on the box for hours, so WLM is pretty much outclassed here. I feel your flatline pain. No 'Dolly Parton' figure (TITanic peaks at 10:00 and 14:00) ;-D Only Table Mountain figure (peak all the time from 06:00 - 18:00) :-D Only way to lower CPU usage is - pull the plug! ;-D We really need PFA in place with the SMF flooding warning for those situations because no one realizes there is a problem most of the time ... PFA? Sorry, which meaning are you using here? AcronymFinder gave me lots of possible definitions ... ;-) Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
On Thu, 21 Jun 2012 23:31:51 -0500, Barbara Nitz nitz-...@gmx.net wrote: This is one of those problems you don't have until you have it if you normally have spare cycles. WLM is funny that way. :-) Yes, and PHBs don't understand that. So whoever screams loudest gets IMP=1 until *everyone* is imp=1. :-( (I know that my frustration is showing...) Reminds me of a joke in the recording studio when doing a mix and to make everything louder than everything else. :-) -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://expertanswercenter.techtarget.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
On Thu, 21 Jun 2012 23:31:51 -0500, Barbara Nitz nitz-...@gmx.net wrote: We've never had a problem keeping up even during heavy periods Then you're lucky. How big is your standard in-storage SMF buffer? Max size - 1G How many DB2s do you run on any one lpar? We went through quite a few hoops to prevent constantly loosing SMF data. The smaller plexes actually have more DB2s and seem to have problems more often from things like accounting traces left on or loops on development regions. There is no DB2 data sharing in those environments. The largest sysplex has DB2 datasharing with the largest 2 DB2 subsystems have regions on each LPAR. There are some other DB2 subsystems that are production (I think) not shared across all LPARs and some QA/DEVL regions either on a single LPAR only or a subset of the sysplex. Mark -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://expertanswercenter.techtarget.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
On 6/22/2012 7:00 AM, Mark Zelden wrote: Reminds me of a joke in the recording studio when doing a mix and to make everything louder than everything else. :-) Or when on-stage and everyone keeps turning up their volume... :-) -- Edward E Jaffe Phoenix Software International, Inc 831 Parkview Drive North El Segundo, CA 90245 310-338-0400 x318 edja...@phoenixsoftware.com http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
On Wed, 20 Jun 2012 21:03:07 +, Campbell Jay james.l.campb...@irs.gov wrote: The solution in this case was stopping type 100 - 102 collection. Changing the Workload Management Class of the copy job to STCHI. The DUMPXY process caught up with itself in about 20 minutes. My SMFDUMP jobs are also in STC SC with a high importance (2). Almost nothing is IMP=1 in my environment.This is one of those problems you don't have until you have it if you normally have spare cycles. WLM is funny that way. :-) We've never had a problem keeping up even during heavy periods, so I still haven't implemented SMF logger other than in sandbox systems (that and the initial support for the dump process etc. would have meant to many changes across a very large environment with many sysplexes). There have been a few occasions were a looping DB2 process or MQ trigger from a devl/qa region made it difficult to keep up, but we did. We really need PFA in place with the SMF flooding warning for those situations because no one realizes there is a problem most of the time until SMF processing happens at night since all the dumps are automatic vie IEFU29. A couple of times there were more than 255 daily GDGs created from offloads and we had to manually code the volsers in MXG JCL. Mark -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://expertanswercenter.techtarget.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
This is just type 100 - 102 records on 2 out of 11 LPARS BTW. For some reason - the word overkill came to mind. So I decided to conduct Operation Overkill. 36 MAN* datasets per LPAR... MAN0 - MANZ... 3337 cylinders per MAN*. 24 hourly jobs to MOD the DUMPXY contents to virtual tape. ( First job each day is DISP=NEW just to keep things pretty ) The following day we have a problem. IEFU29 isseued I SMF every 3 minutes. Here's what 1 of the 2 LPARs looked like... Jay Campbell -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Campbell Jay Sent: Monday, June 11, 2012 6:01 PM To: ibm-m...@bama.ua.edu Subject: SMF volume Is anyone else is required to collect and retain 500,000 cylinders + of SMF data on an LPAR in one day... how do you handle the downstream copies ? Jay Campbell -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Recall: SMF volume - follow-up
Campbell Jay would like to recall the message, SMF volume - follow-up. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
Jay, You may want to check what TRACE classes your DBA are using. There are a few that can generate copious amounts of data into SMF, but they may only need to be captured for short periods of time. Ron From: Campbell Jay james.l.campb...@irs.gov To: IBM-MAIN@listserv.ua.edu Sent: Wed, June 20, 2012 2:03:40 PM Subject: Re: [IBM-MAIN] SMF volume - follow-up This is just type 100 - 102 records on 2 out of 11 LPARS BTW. For some reason - the word overkill came to mind. So I decided to conduct Operation Overkill. 36 MAN* datasets per LPAR... MAN0 - MANZ... 3337 cylinders per MAN*. 24 hourly jobs to MOD the DUMPXY contents to virtual tape. ( First job each day is DISP=NEW just to keep things pretty ) The following day we have a problem. IEFU29 isseued I SMF every 3 minutes. Here's what 1 of the 2 LPARs looked like... 0080 P-SYS1.FJI0.MAN0 SMFDB2100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN1 SMFDB3100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN2 SMFDB2100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN3 SMFDB3100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN4 SMFDB210005091 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN5 SMFDB3100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN6 SMFDB2100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN7 SMFDB3100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN8 SMFDB2100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MAN9 SMFDB3100050 100 DUMP REQUIRED 0080 S-SYS1.FJI0.MANA SMFDB210005024 DUMP REQUIRED 0080 S-SYS1.FJI0.MANB SMFDB310005087 ACTIVE 0080 S-SYS1.FJI0.MANC SMFDB2100050 0 ALTERNATE 0080 S-SYS1.FJI0.MAND SMFDB3100050 0 ALTERNATE 0080 S-SYS1.FJI0.MANE SMFDB2100050 0 ALTERNATE 0080 S-SYS1.FJI0.MANF SMFDB3100050 0 ALTERNATE Hourly copy job coundn't get enough CPU to copy out the data to virtual tape. The solution in this case was stopping type 100 - 102 collection. Changing the Workload Management Class of the copy job to STCHI. The DUMPXY process caught up with itself in about 20 minutes. Jay Campbell -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Campbell Jay Sent: Monday, June 11, 2012 6:01 PM To: ibm-m...@bama.ua.edu Subject: SMF volume Is anyone else is required to collect and retain 500,000 cylinders + of SMF data on an LPAR in one day... how do you handle the downstream copies ? Jay Campbell -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SMF volume - follow-up
Jay, Make certain that SMFPRMxx has DDCONS=NO, NODETAIL, EMPTYEXCPCSEC(SUPPRESS), and INTERVAL is set to at least an hour Then check out for DB2 traces That should cut down your DB2 SMF records Steve -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@listserv.ua.edu] On Behalf Of Campbell Jay Sent: Wednesday, June 20, 2012 4:25 PM To: IBM-MAIN@listserv.ua.edu Subject: Re: SMF volume - follow-up This is just type 100 - 102 records on 2 out of 11 LPARS BTW. For some reason - the word overkill came to mind. So I decided to conduct Operation Overkill. 36 MAN* datasets per LPAR... MAN0 - MANZ... 3337 cylinders per MAN*. 24 hourly jobs to MOD the DUMPXY contents to virtual tape. ( First job each day is DISP=NEW just to keep things pretty ) The following day we have a problem. IEFU29 isseued I SMF every 3 minutes. Here's what 1 of the 2 LPARs looked like... Jay Campbell -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Campbell Jay Sent: Monday, June 11, 2012 6:01 PM To: ibm-m...@bama.ua.edu Subject: SMF volume Is anyone else is required to collect and retain 500,000 cylinders + of SMF data on an LPAR in one day... how do you handle the downstream copies ? Jay Campbell -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
