Re: Turning on additional CPs
This used to be a big deal back in the 3090 (and prior) days. Less so now as the numbers sustain us to very high numbers of engines without much discomfort. In the original case (3 going to 4) I wanted (a few days ago) to point out the engine speed decrease was slight enough to probably not be the cause. I might be wrong :-) but that's still my opinion... I know we've not really talked about MP ratios as a community for a very long time. I'm wondering what's been done in the past 15 years (other than multi-book machines since z990 and Hiperdispatch) to potentially change the picture. Anyone care to chip in? I'm also going to think about the effects of multibook and HD myself. On multibook I think it analogous to the old 280 and 380 models that were faster than the 200 and 300 models despite having the same number of engines (2 and 3 respectively) because of the second lot of cache. Cheers (as I just heard George Clooney say :-) ) Martin Martin Packer, Mainframe Performance Consultant, zChampion Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Edward Jaffe edja...@phoenixsoftware.com To: IBM-MAIN@bama.ua.edu, Date: 12/02/2012 07:54 Subject: Re: Turning on additional CPs Sent by: IBM Mainframe Discussion List IBM-MAIN@bama.ua.edu On 2/9/2012 8:53 AM, Staller, Allan wrote: There is a well know impact of additional CP's known as the MP effect. Going from 1 to 2 engines does not get you twice as much horsepower. Only 1.9 times as much. In the early days 360/65 AP 305/65 MP, the effect was only 1.7 times. Specifically, if you look at z196 machines http://tech-news.com/publib/pl2817.html you'll see the following LSPR ratios: Modl Ratio Delta 701 2.15 N/A 702 4.06 1.91 703 5.92 1.86 704 7.72 1.80 705 9.47 1.75 706 11.17 1.70 707 12.82 1.65 and so on... -- Edward E Jaffe Phoenix Software International, Inc 831 Parkview Drive North El Segundo, CA 90245 310-338-0400 x318 edja...@phoenixsoftware.com http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
On 2/9/2012 8:53 AM, Staller, Allan wrote: There is a well know impact of additional CP's known as the MP effect. Going from 1 to 2 engines does not get you twice as much horsepower. Only 1.9 times as much. In the early days 360/65 AP 305/65 MP, the effect was only 1.7 times. Specifically, if you look at z196 machines http://tech-news.com/publib/pl2817.html you'll see the following LSPR ratios: Modl Ratio Delta 701 2.15 N/A 702 4.06 1.91 703 5.92 1.86 704 7.72 1.80 705 9.47 1.75 706 11.17 1.70 707 12.82 1.65 and so on... -- Edward E Jaffe Phoenix Software International, Inc 831 Parkview Drive North El Segundo, CA 90245 310-338-0400 x318 edja...@phoenixsoftware.com http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
I hate to ask the simple (possibly overlooked) question, but is it possible you moved from a configuration with 3 CPs to a configuration with 4 CPs but with more of a sub-capacity setting? That's certainly possible, at least on a System z10 BC. For example, if you started with a 2098-D03 capacity configuration and moved to a C04, you would be moving from a configuration with a PCI (Processor Capacity Index) of 121 up to a PCI of 130 -- a greater than 7% increase. In other words, in the IBM Large Systems Performance Reference (LSPR) tables, you would be moving to a higher performance system, with greater throughput for typical measured LSPR workloads. However, the uni speed of each engine would be reduced, so you would expect to see somewhat elongated execution time for single threaded workloads relative to the previous configuration. So that's another thing to check: did you increase overall MIPS but decrease per-engine MIPS? On the z9 BC, z10 BC, and z114 models that's very easy to check: just look at the letter in the capacity setting. If the letter didn't change, then you should only see SMP effect. If the letter is lower (D to C, for example), then you probably are experiencing what I am describing. I suppose I should also ask the simple question of whether that additional CP is properly activated, defined, and actually available to your z/OS LPAR (s) for dispatch, but maybe that's been covered by now. :-) Keep in mind that z/OS Workload Manager (WLM) is boss, subject to capacity limits of course. If you have a job that's running longer, but it's still meeting or beating the WLM goal, then z/OS considers that perfectly fine because it's doing exactly what you told it to do. It could very well be that more overall work is getting done faster thanks to the additional engine, and WLM made the correct adjustments based on your settings. (For example, work that was getting deferred is now getting executed, and that particular work is putting some greater stress on the caches and/or on I/O.) If you aren't happy with the results, consider adjusting WLM settings. Timothy Sipples Resident Enterprise Architect (Based in Singapore) E-Mail: timothy.sipp...@us.ibm.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
Thanks for the input Timothy. We have a 4 engine machine, but only had three turned on. We wanted to keep one in our back pocket, but then decided to turn on the 4th to see what it would do. We didn't get the expected results. However, you bring up some good points to consider regarding WLM. I don't think WLM has been tuned in a long time and not really sure if we have anyone that it WLM literate. lol Thanks again. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
Guess I'd start with RMF WKLD and see if anything pops out. Device busy, LCU, Storage unit, paging and work from there. There are people and software to help you tune your system. _www.watsonwalker.com_ (http://www.watsonwalker.com) is a good place to start. Goal Tender will analyze goals vs thruput. In a message dated 2/10/2012 1:47:59 P.M. Central Standard Time, gsg_...@yahoo.com writes: However, you bring up some good points to consider regarding WLM. I don't think WLM has been tuned in a long time and not really sure if we have anyone that it WLM literate. lol -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Turning on additional CPs
We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. TIA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
That is interesting. If the job single threads, then I'd say that a slow down is possible. It's called the MP effect. Basically, in an SMP (Symmetric Multi Processing) environment, where each CPU has equal access to main memory, then you get an interference effect fighting for access to the shared memory. The more CPUs, the more interference. IBM tries to address this by having separate portions of memory be on separate memory busses. But I don't know how the z10's memory is laid out. -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of gsg Sent: Thursday, February 09, 2012 10:39 AM To: IBM-MAIN@bama.ua.edu Subject: Turning on additional CPs We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. TIA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
Yes. Especially if the jobs are CPU bound. Going back to the discussion of dedicated vs. shared CPs. Etc. There is a well know impact of additional CP's known as the MP effect. Going from 1 to 2 engines does not get you twice as much horsepower. Only 1.9 times as much. In the early days 360/65 AP 305/65 MP, the effect was only 1.7 times. Each additional processor increases the accumulated overhead of trying to keep everything in sync between the members of the group. 3 processors would (for example) (1+(1*.9)+(1*.8)) *.9) =2.7, not 3.0. This overhead is a non-linear relationship. The more processors, the greater (relatively speaking) the MP effect. i.e. less horsepower added per additional engine This is the reason IBM invented Hiperdispatch. By treating the processors as members of group (books), the MP effect is reduced and overhead is reduced.. CMG (www.cmg.org), Cheryl Watson (www.watsonwalker.com), Share (www.share.org) and IBM Techdocs (http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/Technotes) are all good sources for additional information. HTH, snip We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. /snip -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
I suppose that is reasonable for a single threaded, CPU bound job as a little is lost from each engine as another is added. However, you should be able to run more concurrent work giving a better over all through put. Another benefit of another engine is that, if not needed for anything else, z/os likes to direct I/O interupts to just one engine. This allows the other engines run a little smoother and again should increase your overall thoughput. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of gsg Sent: Thursday, February 09, 2012 10:39 AM To: IBM-MAIN@bama.ua.edu Subject: Turning on additional CPs We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. TIA -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
But would you really expect an effect of more than a few percentage points in the efficiency of one CP from only going from 3 to 4 engines? If you are seeing a large difference in run time, perhaps you should also look for an explanation that can produce a much larger difference than the MP effect. Maybe by removing a CPU bottleneck you have moved your major system constraint elsewhere, either to real memory or to DASD throughput, or some logical interlock. Perhaps the longer running jobs are now doing significant paging because of greater contention for real memory, or are having to wait on physical I/O to DASD more -- because other things that used to be too starved for CPU to compete are now running and using resources other than CPU that used to be more plentiful in a CPU-starved environment. JC Ewing On 02/09/2012 12:04 PM, Hal Merritt wrote: I suppose that is reasonable for a single threaded, CPU bound job as a little is lost from each engine as another is added. However, you should be able to run more concurrent work giving a better over all through put. Another benefit of another engine is that, if not needed for anything else, z/os likes to direct I/O interupts to just one engine. This allows the other engines run a little smoother and again should increase your overall thoughput. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of gsg Sent: Thursday, February 09, 2012 10:39 AM To: IBM-MAIN@bama.ua.edu Subject: Turning on additional CPs We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. TIA ... -- Joel C. Ewing,Bentonville, AR jcew...@acm.org -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
Re: Turning on additional CPs
Frankly, the increase in run time is not what I would expect at all. But, to answer the OP's question, it is plausible given some specific situations. That said, I agree with you that more questions need to be asked and you presented some very good places to look. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Joel C. Ewing Sent: Thursday, February 09, 2012 2:49 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Turning on additional CPs But would you really expect an effect of more than a few percentage points in the efficiency of one CP from only going from 3 to 4 engines? If you are seeing a large difference in run time, perhaps you should also look for an explanation that can produce a much larger difference than the MP effect. Maybe by removing a CPU bottleneck you have moved your major system constraint elsewhere, either to real memory or to DASD throughput, or some logical interlock. Perhaps the longer running jobs are now doing significant paging because of greater contention for real memory, or are having to wait on physical I/O to DASD more -- because other things that used to be too starved for CPU to compete are now running and using resources other than CPU that used to be more plentiful in a CPU-starved environment. JC Ewing On 02/09/2012 12:04 PM, Hal Merritt wrote: I suppose that is reasonable for a single threaded, CPU bound job as a little is lost from each engine as another is added. However, you should be able to run more concurrent work giving a better over all through put. Another benefit of another engine is that, if not needed for anything else, z/os likes to direct I/O interupts to just one engine. This allows the other engines run a little smoother and again should increase your overall thoughput. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of gsg Sent: Thursday, February 09, 2012 10:39 AM To: IBM-MAIN@bama.ua.edu Subject: Turning on additional CPs We have a z10 with 4 engines. Since upgrading to this box, we were only running 3 engines. However, we recently turned on the 4th engine. We noticed that several jobs started running longer, which we didn't expect. Could turning on additional engines actually make a job run longer? Also, where can I find any read material on the affect of turning on/off engines. TIA ... -- Joel C. Ewing,Bentonville, AR jcew...@acm.org -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
