Re: HMC Management Best Practices
Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? 1. Why do you IPL at all? 2. Why don't you perform IPL by yourself? Hint: remote access to HMC. 3. What's the problem with the operators? I can see the following possibilities: - Nobody's perfect, they simply made a mistake, the situation never happened again. - They can't read, don't understand, are drunk... - They don't have procedures and clearly defined tasks (i.e. using procedure ABC, IPL system XYZ on LPAR 123) -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2008 r. kapita zakadowy BRE Banku SA wynosi 118.642.672 zote i zosta w caoci wpacony. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I believe the writer statement concerning: What are some best practices that you use to prevent these and other? Operator errors while performing HMC tasks? Was not directed only at Operators in the computer room, but instead at those personnel (systems and operations) who operator the HMC. Another Best practice is to publish a configuration sheet that has the current IPL information on it. I like to have these in the HMC book at the console or tape to the wall above some how. Kevin -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of R.S. Sent: Monday, October 06, 2008 12:29 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? 1. Why do you IPL at all? 2. Why don't you perform IPL by yourself? Hint: remote access to HMC. 3. What's the problem with the operators? I can see the following possibilities: - Nobody's perfect, they simply made a mistake, the situation never happened again. - They can't read, don't understand, are drunk... - They don't have procedures and clearly defined tasks (i.e. using procedure ABC, IPL system XYZ on LPAR 123) -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2008 r. kapita zakadowy BRE Banku SA wynosi 118.642.672 zote i zosta w caoci wpacony. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html This e-mail message and any attachments transmitted with it are confidential and are intended solely for the use of its authorized recipient(s). If you are not an intended or authorized recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the information contained in this e-mail is prohibited. If you have received this message in error or are not authorized to receive it, please immediately notify the sender and delete the original message and all copies of it from your computer. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
No, I don't. - Robert B. Richards(Bob) US Office of Personnel Management 1900 E Street NW Room: BH04L Washington, D.C. 20415 Phone: (202) 606-1195 Email: [EMAIL PROTECTED] - -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Thursday, October 02, 2008 1:46 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices Have you got Hercules ready to the point where you can handle AWS tapes? I can try to generate a copy of what Sam sent me in AWSTAPE format from your tape. Rick Richards, Robert B. wrote: I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Once again, I apologize for not noticing that this was going back to the list. Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Richards, Robert B. Sent: Friday, October 03, 2008 8:25 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices No, I don't. - Robert B. Richards(Bob) US Office of Personnel Management 1900 E Street NW Room: BH04L Washington, D.C. 20415 Phone: (202) 606-1195 Email: [EMAIL PROTECTED] - -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Thursday, October 02, 2008 1:46 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices Have you got Hercules ready to the point where you can handle AWS tapes? I can try to generate a copy of what Sam sent me in AWSTAPE format from your tape. Rick -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I know how to lock images, but I can't seem to figure out how to lock profiles. Thanks. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I don't know about best practices, but we: - Make religious use of LPAR grouping - Test images in a test folder, Prod images in a Prod folder. That at least keeps the Operators from activating a Prod image when they meant to activate a test image. We still get the occasional brain check where the wrong LPAR was activated in the middle of a change window. That can be fun. (NOT) - Lock everything. However, experience has shown that once an operator gets used to the extra step, that operator can, but not necessarily will, get numb to it. - Make religious use of activation profiles. We would like to use the dynamic settings approach described earlier, but we that responsibility would still fall in the hands of Operations. - Practice, practice, practice. Our Operators IPL our sandbox systems at least as often as our sysprogs do these days. - Train, coach, and most of all - communicate. Every request is confirmed with the originator and on-the-spot peer-reviewed with another Operator. It takes an extra minute or two, but I think it's worth the effort. We have an extra body in on change weekends just to run point. - Have a system of checks and balances. Only Operations is responsible for updating profiles, weights, etc., but they do it with information provided by sysprogs, performance, storage, etc., and everything is cross-checked before anything is updated. There's always one in the bunch who takes a little more patience and tolerance than the rest of the group, but that's true of sysprogs, storage managers, and performance analysts, too. Hopefully, appropriate use of PR's will address the situation if nothing else seems to work. I also had one instance where the entire group kept me up nights (literally... the all-too- frequent 3AM call - were getting message IEAxxxI and we've never seen it before, what does it mean?). That meant time for me to seek alternate employment. FWIW, Art Gutowski Ford Motor Company ITInfrastructure -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Upon second look SMP told me that the PTF was accepted, it is in the TLIB as well as in the DLIB. So, it now is in all three zones. I don't want to reject the PTF from TLIB/DLIB just from the global zone. How would I handle this situation? Thanks Richards, Robert B. [EMAIL PROTECTED] 10/2/2008 10:01 AM I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html _ LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately, then delete this message and empty from your trash. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Howard, The REJECT command only removes PTFs from the GLOBAL zone. If you want to affect the TLIB/DLIB you would use RESTORE. I use RESTORE/RECEIVE process when I am working with a usermod, it has not touched the TLIB/DLIB when I do this. Lizette Upon second look SMP told me that the PTF was accepted, it is in the TLIB as well as in the DLIB. So, it now is in all three zones. I don't want to reject the PTF from TLIB/DLIB just from the global zone. How would I handle this situation? Thanks -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Have you got your CD-ROM yet ?? Richards, Robert B. wrote: I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Rick, Yes, but after looking through it, it contained that exact same information as the last one you sent me. It is not a copy of the files from the tape I sent you that you sent to Sam. Listening to a 1.10 webcast at the moment. I'll get in touch with you soon (not today). Thanks for sending it though. Bob - Robert B. Richards(Bob) US Office of Personnel Management 1900 E Street NW Room: BH04L Washington, D.C. 20415 Phone: (202) 606-1195 Email: [EMAIL PROTECTED] - -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Thursday, October 02, 2008 11:42 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices Have you got your CD-ROM yet ?? Richards, Robert B. wrote: I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
My apologies to the list for the private email to Rick. Bob -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
On Thu, 2 Oct 2008 11:25:14 -0400, Howard Rifkind [EMAIL PROTECTED] wrote: Upon second look SMP told me that the PTF was accepted, it is in the TLIB as well as in the DLIB. So, it now is in all three zones. I don't want to reject the PTF from TLIB/DLIB just from the global zone. How would I handle this situation? Thanks == SETBOUNDARY (GLOBAL). REJECT SELECT ( sysmod ) BYPASS(APPLYCHECK,ACCEPTCHECK) . But this tells me you probably don't have you cleanup options set to delete the MCS from the global zone as I mentioned in my last post. So you may want to run a REJECT PURGE in mass mode which will get rid of all the PTFs in your global zone that have been accepted or ones that have been superseded if their superseding PTF was accepted. SETBOUNDARY (GLOBAL). REJECT PURGE (dlibzone). Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
In addition to what Mark mentions, also optionally setup your SMPE options to include a RECEIVE ZONE GROUP, so that PTF's once applied and accepted, don't get re-received again if you order bulk maintenance. _ Dave Jousma Assistant Vice President, Mainframe Services [EMAIL PROTECTED] 1830 East Paris, Grand Rapids, MI 49546 MD RSCB1G p 616.653.8429 f 616.653.8497 -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Mark Zelden Sent: Thursday, October 02, 2008 11:51 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices On Thu, 2 Oct 2008 11:25:14 -0400, Howard Rifkind [EMAIL PROTECTED] wrote: Upon second look SMP told me that the PTF was accepted, it is in the TLIB as well as in the DLIB. So, it now is in all three zones. I don't want to reject the PTF from TLIB/DLIB just from the global zone. How would I handle this situation? Thanks == SETBOUNDARY (GLOBAL). REJECT SELECT ( sysmod ) BYPASS(APPLYCHECK,ACCEPTCHECK) . But this tells me you probably don't have you cleanup options set to delete the MCS from the global zone as I mentioned in my last post. So you may want to run a REJECT PURGE in mass mode which will get rid of all the PTFs in your global zone that have been accepted or ones that have been superseded if their superseding PTF was accepted. This e-mail transmission contains information that is confidential and may be privileged. It is intended only for the addressee(s) named above. If you receive this e-mail in error, please do not read, copy or disseminate it in any manner. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of this information is prohibited. Please reply to the message immediately by informing the sender that the message was misdirected. After replying, please erase it from your computer system. Your assistance in correcting this error is appreciated. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
On Thu, 2 Oct 2008 11:55:49 -0400, Jousma, David [EMAIL PROTECTED] wrote: In addition to what Mark mentions, also optionally setup your SMPE options to include a RECEIVE ZONE GROUP, so that PTF's once applied and accepted, don't get re-received again if you order bulk maintenance. Good advise. Or you can also add ZONEGROUP(ALLZONES) to your RECEIVE command. However, both make the receive process longer since it has to look at each sysmod and determine if it should really be received or not. Pay me now, or pay me later with mass reject. This is were SMP/E 3.4 and above RECEIVE ORDER (or the prior report method) comes in handy since it only orders what you don't have to begin with. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Have you got Hercules ready to the point where you can handle AWS tapes? I can try to generate a copy of what Sam sent me in AWSTAPE format from your tape. Rick Richards, Robert B. wrote: I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Very sorry about this ... in a rush and picked the wrong email to respond to. Howard Rifkind [EMAIL PROTECTED] 10/2/2008 11:25 AM Upon second look SMP told me that the PTF was accepted, it is in the TLIB as well as in the DLIB. So, it now is in all three zones. I don't want to reject the PTF from TLIB/DLIB just from the global zone. How would I handle this situation? Thanks Richards, Robert B. [EMAIL PROTECTED] 10/2/2008 10:01 AM I'm sure Bob Richards remembers The Chairbreaker! :-) I'm trying to forget! grin Bob -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Rick Fochtman Sent: Wednesday, October 01, 2008 4:18 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html _ LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately, then delete this message and empty from your trash. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html _ LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately, then delete this message and empty from your trash. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
HMC Management Best Practices
One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? -- Mark Jacobs Time Customer Service Tampa, FL Today, we celebrate the first glorious anniversary of the Information Purification Directives. We have created, for the first time in all history, a garden of pure ideology. Where each worker may bloom secure from the pests of contradictory and confusing truths. Our Unification of Thoughts is more powerful a weapon than any fleet or army on earth. We are one people, with one will, one resolve, one cause. Our enemies shall talk themselves to death and we will bury them with their own confusion. We shall prevail! Apple's television commercial - Super Bowl - 1984 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? Education Threat of termination Use of staff more sophisticated than systems operators Locking of HMC Change control/auditing - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Umm, train them? Then use test LPAR's to keep their skills fresh? And operators should NOT be changing weights, storage, CPU's etc. without a sysprog there. Define the CP's and use config to vary them on/off. Just my $.02 Doug Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I tend to agree with Doug. The only thing our operators ever do is IPL. They do have to select the correct LPAR and put in the correct load address and loadparm if that changes. The loadparm rarely changes... usually when we upgrade the OS it temporarily points to LOADx9 for example... for z/OS 1.9. There have been times where we ask them to do an activate because memory has been added, but that usually happens on a weekend where a sysprog is around and we do it anyway. Two things that we do that may help if you don't already do it: 1) Lock all LPARs (we lock all but the sandbox LPARs since we IPL them all the time). That way it makes someone think twice as they unlock the lpar and IPL it. 2) Consider making different groups depending on how big your environment is. In addition to several IPL icon groups, we have a separate SADUMP groups / icons. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html On Wed, 1 Oct 2008 09:21:29 -0400, Doug Fuerst [EMAIL PROTECTED] wrote: Umm, train them? Then use test LPAR's to keep their skills fresh? And operators should NOT be changing weights, storage, CPU's etc. without a sysprog there. Define the CP's and use config to vary them on/off. Just my $.02 Doug Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Hi Mark, In respect of your request for information on: What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? Having had some experience with a company managing 100+ z/OS Systems, across 7+ sites I think the two staple requirements to avoid problems at IPL are training, and adequate documentation. Automation is useful, but can also be counter-productive at times as it can dumb things down and detract from the ability to gain experience. Personally I am not in favour of the 'Let the System Programmer' do it. My preference, even when a Technical Services Manager, was for operations to have the required training, to perform their allotted tasks. This was backed up by good operating procedures, and change control records stipulating what needed doing, especially when it deviated from the norm. In the environments that I am used to, the Operations team were the guardians of service in that they were the first butt to be kicked. It was appropriate that they had the means of managing their environment albeit with technical backup for systems and applications as required. I guess at the time our operators were lucky to maintain their experience due to the high volume of changes requiring a system outage, at least one per system per month. (Typically engineered to re-ipl all images (lpars) on the same complex. If that luxury is not available then the ability to practice different scenarios on a training lpar would be worth investigating if resource could be spared. In the context of costing this in, it should be weighed against the cost of the unscheduled disruption caused by the current errant processes. I am sorry I can offer nothing more concrete but hopefully it is fuel for thought. Obviously if a particular individual is the culprit in a number of incidents then it does merit a review of whether they are appropriately employed. Kind regards - Terry Terry Sambrooks Director KMS-IT Limited 228 Abbeydale Road South Dore, Sheffield, S17 3LA, UK Tel: +44 (0)114 262 0933 WEB: www.legac-e.co.uk Company Reg: 3767263 at the above address All outgoing E-mail is scanned, but it remains the recipient's responsibility to ensure their system is protected from spy-ware, trojans, viruses, and worms. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
snip proper use of the HMC by the operators when they perform their job responsibilities unsnip I tend to agree that 'their responsibility' appears to be more than operators tend to have. Another option is to support remote HMC access, inside a VPN(ish) environment, and let the sysprog(s) tend to non IPL procedures. But do lock the non sandbox partitions. Jack Kelly 202-502-2390 (Office) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
-Original Message- From: IBM Mainframe Discussion List On Behalf Of Mark Jacobs One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? George Foreman holding a clue-by-four. Training and practice, attention to detail. -jc- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Let the operator do the IPLs with only one Default LOAD profile that uses dynamically changed address for Load address and Use dynamically changed parameter for the load parameters. The system programmers will then control everything via HCD. Of course the operators still need to perform the actions on the correct LPAR. Regards, Jihad K. Kawkabani IT Systems Engineer Consultant Voice: 440.395.0740 Network: 575.0740 Cell: 440.465.2969 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Mark, I've found that training and frequent usage of the HMC reducing confusions. However some safe guards would be: 1. Change the PASSWORDS for all users other than OPERATOR. 2. Lock the IPL Profiles. Forces a DO YOU REALLY WANT TO IPL PROD moment. 3. Prepare HMC Documents and procedures. 4. Allow changes of the LOAD ADDRESSES and LOAD PARMS only. Kevin -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Mark Jacobs Sent: Wednesday, October 01, 2008 9:11 AM To: IBM-MAIN@BAMA.UA.EDU Subject: HMC Management Best Practices One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? -- Mark Jacobs Time Customer Service Tampa, FL Today, we celebrate the first glorious anniversary of the Information Purification Directives. We have created, for the first time in all history, a garden of pure ideology. Where each worker may bloom secure from the pests of contradictory and confusing truths. Our Unification of Thoughts is more powerful a weapon than any fleet or army on earth. We are one people, with one will, one resolve, one cause. Our enemies shall talk themselves to death and we will bury them with their own confusion. We shall prevail! Apple's television commercial - Super Bowl - 1984 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html This e-mail message and any attachments transmitted with it are confidential and are intended solely for the use of its authorized recipient(s). If you are not an intended or authorized recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the information contained in this e-mail is prohibited. If you have received this message in error or are not authorized to receive it, please immediately notify the sender and delete the original message and all copies of it from your computer. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Jihad K Kawkabani wrote: Let the operator do the IPLs with only one Default LOAD profile that uses dynamically changed address for Load address and Use dynamically changed parameter for the load parameters. The system programmers will then control everything via HCD. Of course the operators still need to perform the actions on the correct LPAR. Regards, Jihad K. Kawkabani IT Systems Engineer Consultant Voice: 440.395.0740 Network: 575.0740 Cell: 440.465.2969 Can you expand on how that gets setup both in the HMC and HCD world? -- Mark Jacobs Time Customer Service Tampa, FL Today, we celebrate the first glorious anniversary of the Information Purification Directives. We have created, for the first time in all history, a garden of pure ideology. Where each worker may bloom secure from the pests of contradictory and confusing truths. Our Unification of Thoughts is more powerful a weapon than any fleet or army on earth. We are one people, with one will, one resolve, one cause. Our enemies shall talk themselves to death and we will bury them with their own confusion. We shall prevail! Apple's television commercial - Super Bowl - 1984 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
First, I would respectfully submit that this not an operator problem. Operators make mistakes and any dependence on operators accepts that as a consequence. To eliminate the mistakes, you must eliminate the operators. That said, I have a few tricks that help minimize my screw ups: 1. You can lock out 'disruptive' LPAR changes. I like to keep my production LPARs locked. 2. Use only activation profiles for ordinary IPL's. 3. Changes to an activation profile are made by a qualified individual. I am told that activating a profile is the same as a deactivate/activate/IPL sequence. For us, that does not add any noticeable time to the IPL sequence and eliminates a decision point. Plus, there is assurance that LPAR changes are picked up. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Mark Jacobs Sent: Wednesday, October 01, 2008 8:11 AM To: IBM-MAIN@BAMA.UA.EDU Subject: HMC Management Best Practices One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... What are some best practices that you use to prevent these and other operator errors while performing HMC tasks? -- Mark Jacobs Time Customer Service Tampa, FL NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
On Wed, 1 Oct 2008 09:25:56 -0500, Hal Merritt wrote: First, I would respectfully submit that this not an operator problem. Operators make mistakes and any dependence on operators accepts that as a consequence. People make mistakes. Operators certainly are in that category. Operators can be trained to do the job correctly, just as sysprogs can. To eliminate the mistakes, you must eliminate the operators. That said, I have a few tricks that help minimize my screw ups: 1. You can lock out 'disruptive' LPAR changes. I like to keep my production LPARs locked. 2. Use only activation profiles for ordinary IPL's. 3. Changes to an activation profile are made by a qualified individual. You are right back where you started. People make mistakes. If operators are properly trained, they can certainly make these kinds of changes as well as anyone. Indeed, I would argue that the opportunities for error are reduced if all HMC operations are performed by the same people, rather than spread some of the tasks among different groups. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
But the OP is complaining that that strategy is not working. And that experience/observation is in line with my own. That is, even trained, experienced operators are making an unacceptable number of mistakes. We have to drill down to find root causes. Here I think the most reasonable attack is to simplify what the operators are expected to do. And that means shifting the responsibility for making critical system changes to a sysprog. With a combination of locking most loved LPARS and reducing the operator involvement to clicking on an 'activate' icon, I would daresay the error rate would drop to near zero. No process is perfect, but this basic strategy has worked for me in the past. A key to success would be an unchanging operator script with absolute minimal use of the word 'if'. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Tom Marchant Sent: Wednesday, October 01, 2008 10:57 AM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices On Wed, 1 Oct 2008 09:25:56 -0500, Hal Merritt wrote: ..snip You are right back where you started. People make mistakes. If operators are properly trained, they can certainly make these kinds of changes as well as anyone. Indeed, I would argue that the opportunities for error are reduced if all HMC operations are performed by the same people, rather than spread some of the tasks among different groups. -- Tom Marchant NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I am not sure how you define your LOAD profile. What I do is I use the DEFAULTLOAD profile supplied by IBM and then check the two boxes: Use dynamically chaged address across from the Load address box. Use dynamically changed parameter across from the load parameter box. In HCD: Make sure in your Processor definition you specify in SNA Address a Network Name and a CPC name. Then from the HCD Option 2 (Activate or Process Configuration Data) pick option 11 (Build and manage S/390 microprocessor IOCDSs and IPL attributes) panel. Enter an i next to the CPC you want to manage you will get a screen similar to: ---Next IPLPARM Processor Partition Next IPLADDR IODFLOADxx Prompt/Msg Nucleus ID Name IPL DeviceDevice Suffix Option Suffix IBMX SYSICF __ _ _ IBMX SYS1 2F90 2C1C90 M 1 IBMX SYS2 2F90 2C1C90 M 1 IBMX SYS3 2F90 2C1C90 M 1 Enter your parameters in the fields. Of course there are some security requirements when MVS is under RACF or whatever your security product's control to protect them. Regards, Jihad K. Kawkabani IT Systems Engineer Consultant Voice: 440.395.0740 Network: 575.0740 Cell: 440.465.2969 Mark Jacobs [EMAIL PROTECTED] SERV.COM To Sent by: IBM IBM-MAIN@BAMA.UA.EDU Mainframe cc Discussion List [EMAIL PROTECTED] Subject .EDU Re: HMC Management Best Practices 10/01/2008 10:17 AM Please respond to IBM Mainframe Discussion List [EMAIL PROTECTED] .EDU Jihad K Kawkabani wrote: Let the operator do the IPLs with only one Default LOAD profile that uses dynamically changed address for Load address and Use dynamically changed parameter for the load parameters. The system programmers will then control everything via HCD. Of course the operators still need to perform the actions on the correct LPAR. Regards, Jihad K. Kawkabani IT Systems Engineer Consultant Voice: 440.395.0740 Network: 575.0740 Cell: 440.465.2969 Can you expand on how that gets setup both in the HMC and HCD world? -- Mark Jacobs Time Customer Service Tampa, FL Today, we celebrate the first glorious anniversary of the Information Purification Directives. We have created, for the first time in all history, a garden of pure ideology. Where each worker may bloom secure from the pests of contradictory and confusing truths. Our Unification of Thoughts is more powerful a weapon than any fleet or army on earth. We are one people, with one will, one resolve, one cause. Our enemies shall talk themselves to death and we will bury them with their own confusion. We shall prevail! Apple's television commercial - Super Bowl - 1984 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
On Wed, 1 Oct 2008 11:47:18 -0500, Hal Merritt wrote: But the OP is complaining that that strategy is not working. And that experience/observation is in line with my own. The op listed two problems. On Wed, 1 Oct 2008 09:11:17 -0400, Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... Number 1 is clearly operations responsibility. Surely you are not suggesting that this should be done by a sysprog? That is, even trained, experienced operators are making an unacceptable number of mistakes. We have to drill down to find root causes. Sure. Everyone makes mistakes. I've done it. Haven't you? In my last week as the lead systems programmer at a previous job, I IPLed a production LPAR by mistake. I had 30 years experience at the time. Here I think the most reasonable attack is to simplify what the operators are expected to do. And that means shifting the responsibility for making critical system changes to a sysprog. With a combination of locking most loved LPARS and reducing the operator involvement to clicking on an 'activate' icon, I would daresay the error rate would drop to near zero. No process is perfect, but this basic strategy has worked for me in the past. A key to success would be an unchanging operator script with absolute minimal use of the word 'if'. It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- Edward E Jaffe Phoenix Software International, Inc 5200 W Century Blvd, Suite 800 Los Angeles, CA 90045 310-338-0400 x318 [EMAIL PROTECTED] http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I came up as an Operator and I have been a sysprog for a number of years now. All of my fellow sysprogs here were Operators at one time. I suspect that is true for most of us. I remember, when I started, that we were lucky if at least one of our systems didn't turn toes up and crash at least once during the week. Lots of opportunity for hands-on hot seat training. In those days, in my shop, the IPL procedures were a half page laundry list of things to start. No commands, no expected messages, just the list. The theory being that if you didn't know all of the commands and dependences that you weren't qualified to IPL. Three years later, when I was a lead and responsible for training Operators, I could schedule a training change and use all of the leftover time in the 4 hour change window for training. Not just straight up and down training, but I would 'salt' the training with problems for the Operator to recognize, troubleshoot, determine the impact, etc. These days, we have to minimize or eliminate outages, even during the service window. I have had ONE production IPL this year. Let's face, in most shops, Operators just don't IPL nearly as much as we used to. We don't have a big enough box to give the Operators a sandbox. So we document a lot more fully. We have at least two people, at minimum 1 experienced Operator and a green one, even for minor changes. Anything major, a sysprog comes in. We involve Operations in IPLs, etc on our sysprog test bed as much as we can. I came in on the IPL in May. I supported, I explained, I pointed things out, and generally added on as much training as I could for them. It helps. Mistakes happen and people who are afraid to make them generally don't grow in expertise. Still, when dealing with a slacker, or someone who repeatedly fails to prepare, I don't have much tolerance for that. As for IPLing the wrong LPAR, I have seen that happen. One easy mistake is leaving the LPAR icon highlighted after the LOAD. It's really the kind of mistake the 'rusty' and the green fall into. Then at the next IPL of a different LPAR, the leftover highlighted one comes along for the ride. I have found that if I give the procedures to a greenie or a rusty (asking their help to test the procedures), and then I partner with an experienced operator and have the operators show me what to do, I can spot places where procedures, methods, training, etc. need to be improved. That also keeps them involved in the improvement process and in the partnership. That helps to keep a solution focus instead of a finger pointing/blame focus. Involvement in the process improves skill levels, and as Ed said, deepens the bench. That means a more productive working relationship too. We tried automation several years ago. It made things much worse so we pulled it out. Linda Mooney -- Original message -- From: Edward Jaffe [EMAIL PROTECTED] Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- Edward E Jaffe Phoenix Software International, Inc 5200 W Century Blvd, Suite 800 Los Angeles, CA 90045 310-338-0400 x318 [EMAIL PROTECTED] http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
That's true, until you get an operator that's too lazy tocrack open a manual, whether to look up a message or try and learn something new, and he's also too gutless to accept any responsibilities. And a management team that's too soft-hearted (or soft-headed) to do anything about it. BTDTGTSS! I'm sure Bob Richards remembers The Chairbreaker! :-) Edward Jaffe wrote: Tom Marchant wrote: It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. Exactly. The deeper the bench, the better the team. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I saw only the one problem: We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... I've also seen that even with trained, experienced operators to include myself. Training and experience go a long way, but what about when it is not enough? You rework the process. Tough love, some pain, whatever it takes. To answer your question: yes. At least in our tiny shop, the sysprogs do the rare IPL. YMMV If you want to reduce the operator error potential/rate, reduce the number of keystrokes/clicks. Whatever it takes to do that. Not reduce responsibilities mind you, just keystrokes/clicks. -Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Tom Marchant Sent: Wednesday, October 01, 2008 1:32 PM To: IBM-MAIN@BAMA.UA.EDU Subject: Re: HMC Management Best Practices On Wed, 1 Oct 2008 11:47:18 -0500, Hal Merritt wrote: But the OP is complaining that that strategy is not working. And that experience/observation is in line with my own. The op listed two problems. On Wed, 1 Oct 2008 09:11:17 -0400, Mark Jacobs wrote: One of our recurring problems is with the management, i.e. proper use of the HMC by the operators when they perform their job responsibilities; 1) IPL an lpar with a specific load address/load parm. 2) Change lpar settings, storage, cpu's, weights,... We have had many instances of wrong lpars being deactivated and then ipled incorrectly, changes to ipl environments not being applied correctly... Number 1 is clearly operations responsibility. Surely you are not suggesting that this should be done by a sysprog? That is, even trained, experienced operators are making an unacceptable number of mistakes. We have to drill down to find root causes. Sure. Everyone makes mistakes. I've done it. Haven't you? In my last week as the lead systems programmer at a previous job, I IPLed a production LPAR by mistake. I had 30 years experience at the time. Here I think the most reasonable attack is to simplify what the operators are expected to do. And that means shifting the responsibility for making critical system changes to a sysprog. With a combination of locking most loved LPARS and reducing the operator involvement to clicking on an 'activate' icon, I would daresay the error rate would drop to near zero. No process is perfect, but this basic strategy has worked for me in the past. A key to success would be an unchanging operator script with absolute minimal use of the word 'if'. It should be no surprise that with a philosophy of removing responsibility from operators that they would act irresponsibly. My experience is that operators who are treated with respect and given responsibility do good work. Attempts to make a shop operator-proof don't work as well, IMO. -- Tom Marchant NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: HMC Management Best Practices
I'm a small shop (1 CEC, 4 LPARs + Sandbox) so life is a little easier. Major changes a Sysprog is on site to (at least) verify the changes were correct. Sysprog handles all updates to the HMC (we don't use dynamic IPL parm changes) we just update the LOAD profile as needed (maybe once a year). Operators handle all IPL's (really just an Activate). We limited the exposure of multiple LPARs being selected by creating a separate Group on the HMC for each LPAR (a group of one). You can't select multiple groups at a time. Operators instructed to stay away from 'All CPC' and 'All Images'. I would prefer an 'enlightened' operator but realistically there is not enough activity to get the experience. You can't really 'play' with a production CEC and we have little to no need to do anything other than Activate/System Reset. IPL is mostly handled by automation (starting the various tasks) and even automation to tell you what didn't automatically come up! Even have a PFK set up the respond WARM,NOREQ to JES (yet I still got called in one weekend because of a manual reply of WAR,NOREC). Back in the day (before most IPL prompts eliminated we had a PFK set up with stacked responses. Instructions were 'When the console beeps, hit PF1 and enter, next beep hit enter, repeat until no more beeps'. Had some fun IPL'ing with my back turned to the console to prove it worked and was better than keying in the responses. Not that they couldn't but finger checks would occasionally creep up at the wrong time with a call at zero dark thirty for an answer to a never before seen message. Ken Porowski AVP Systems Software CIT Group E: [EMAIL PROTECTED] -Original Message- Linda Mooney These days, we have to minimize or eliminate outages, even during the service window. I have had ONE production IPL this year. Let's face, in most shops, Operators just don't IPL nearly as much as we used to. We don't have a big enough box to give the Operators a sandbox. So we document a lot more fully. We have at least two people, at minimum 1 experienced Operator and a green one, even for minor changes. Anything major, a sysprog comes in. We involve Operations in IPLs, etc on our sysprog test bed as much as we can. I came in on the IPL in May. I supported, I explained, I pointed things out, and generally added on as much training as I could for them. It helps. Mistakes happen and people who are afraid to make them generally don't grow in expertise. Still, when dealing with a slacker, or someone who repeatedly fails to prepare, I don't have much tolerance for that. As for IPLing the wrong LPAR, I have seen that happen. One easy mistake is leaving the LPAR icon highlighted after the LOAD. It's really the kind of mistake the 'rusty' and the green fall into. Then at the next IPL of a different LPAR, the leftover highlighted one comes along for the ride. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
