Re: SORT ando MEMLIMIT best practice
In of87d06c78.43f4a289-on80257cc2.002a9f6c-80257cc2.002c3...@uk.ibm.com, on 04/22/2014 at 09:02 AM, Martin Packer martin_pac...@uk.ibm.com said: VIO has, in any case, been seen as CPU expensive. Because it's simulating a device. I would, however, quite like to see VIO in Memory reborn - with a huge (EAV) device type. Why? I'd much rather have memory mapped[B|P|Q]SAM support and cut out the extra CKD simulation (on top of the CKD simulation in the DASD subsytem.) -- Shmuel (Seymour J.) Metz, SysProg and JOAT ISO position; see http://patriot.net/~shmuel/resume/brief.html We don't care. We don't have to care, we're Congress. (S877: The Shut up and Eat Your spam act of 2003) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
I don't disagree: I didn't really talk implementation but I want to see very large temp data sets in memory, controlled via DFSMS and not too expensive. First step is very large IMHO but recast to be cheaper in CPU terms at the same time would be welcome. Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Shmuel Metz (Seymour J.) shmuel+ibm-m...@patriot.net To: IBM-MAIN@listserv.ua.edu Date: 23/04/2014 12:20 Subject:Re: SORT ando MEMLIMIT best practice Sent by:IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu In of87d06c78.43f4a289-on80257cc2.002a9f6c-80257cc2.002c3...@uk.ibm.com, on 04/22/2014 at 09:02 AM, Martin Packer martin_pac...@uk.ibm.com said: VIO has, in any case, been seen as CPU expensive. Because it's simulating a device. I would, however, quite like to see VIO in Memory reborn - with a huge (EAV) device type. Why? I'd much rather have memory mapped[B|P|Q]SAM support and cut out the extra CKD simulation (on top of the CKD simulation in the DASD subsytem.) -- Shmuel (Seymour J.) Metz, SysProg and JOAT ISO position; see http://patriot.net/~shmuel/resume/brief.html We don't care. We don't have to care, we're Congress. (S877: The Shut up and Eat Your spam act of 2003) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
Possibly the EC12's Flash memory, which come in huge chunks of relatively cheap memory, can help speed up the process of having dozens TBs of temporary datasets in memory. Kees. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Martin Packer Sent: Wednesday, April 23, 2014 13:42 To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: SORT ando MEMLIMIT best practice I don't disagree: I didn't really talk implementation but I want to see very large temp data sets in memory, controlled via DFSMS and not too expensive. First step is very large IMHO but recast to be cheaper in CPU terms at the same time would be welcome. Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Shmuel Metz (Seymour J.) shmuel+ibm-m...@patriot.net To: IBM-MAIN@listserv.ua.edu Date: 23/04/2014 12:20 Subject:Re: SORT ando MEMLIMIT best practice Sent by:IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu In of87d06c78.43f4a289-on80257cc2.002a9f6c-80257cc2.002c3...@uk.ibm.com, on 04/22/2014 at 09:02 AM, Martin Packer martin_pac...@uk.ibm.com said: VIO has, in any case, been seen as CPU expensive. Because it's simulating a device. I would, however, quite like to see VIO in Memory reborn - with a huge (EAV) device type. Why? I'd much rather have memory mapped[B|P|Q]SAM support and cut out the extra CKD simulation (on top of the CKD simulation in the DASD subsytem.) -- Shmuel (Seymour J.) Metz, SysProg and JOAT ISO position; see http://patriot.net/~shmuel/resume/brief.html We don't care. We don't have to care, we're Congress. (S877: The Shut up and Eat Your spam act of 2003) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
-Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Paul Gilmartin Sent: Friday, April 18, 2014 16:31 To: IBM-MAIN@LISTSERV.UA.EDU Subject: SORT ando MEMLIMIT best practice o What are the consequences of allocating SORTWKn to VIO? DFSORT refuses SORTWKs in VIO. Kees. For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
VIO has, in any case, been seen as CPU expensive. Because it's simulating a device. I would, however, quite like to see VIO in Memory reborn - with a huge (EAV) device type. Full(er) disclosure: I wrote a presentation on managing VIO with DFSMS back in 1988. Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Vernooij, CP (SPLXM) - KLM kees.verno...@klm.com To: IBM-MAIN@listserv.ua.edu Date: 22/04/2014 07:40 Subject:Re: SORT ando MEMLIMIT best practice Sent by:IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Paul Gilmartin Sent: Friday, April 18, 2014 16:31 To: IBM-MAIN@LISTSERV.UA.EDU Subject: SORT ando MEMLIMIT best practice o What are the consequences of allocating SORTWKn to VIO? DFSORT refuses SORTWKs in VIO. Kees. For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Mon, 21 Apr 2014 23:03:24 +0100, Martin Packer martin_pac...@uk.ibm.com wrote: I think the term pointer compression is relevant here - to java heap just above 2GB. Yes. Very clever. I was trying to find a link to post and found this: http://www.eecg.toronto.edu/~steffan/workshops/08/cdp/ramarao.ppt I didn't realize it was being done on other platforms also. Mark -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS ITIL v3 Foundation Certified mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://search390.techtarget.com/ateExperts/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Tue, 22 Apr 2014 08:46:40 -0500, Mark Zelden wrote: On Mon, 21 Apr 2014 23:03:24 +0100, Martin Packer wrote: I think the term pointer compression is relevant here - to java heap just above 2GB. Yes. Very clever. ... I didn't realize it was being done on other platforms also. Linux (on x86_64) has been doing it forever - goes even further, allowing reference to a 32 Gig heap rather than just 4 Gig. It uses what we would call double word alignment, so drops the (always zero) bottom 3 bits for offset calculations. Haven't looked at how zLinux works. Shane ... -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Mon, 21 Apr 2014 07:01:50 -0500, Tom Marchant wrote: On Fri, 18 Apr 2014 14:07:29 -0500, Paul Gilmartin wrote: Are there separate pools of real storage for above the bar and below the bar? Pools? No. Pools are a software concept. Real storage with real addresses 2 GiB are below the bar. Real storage with real addresses 2 GiB are above the bar. ITYM = 2 GiB But what do you call real storage with a real address = 3 GiB or thereabouts? I understand that for historic reasons, pertaining only to z/OS, not other OSes, 2 GiB = virtual address 4GiB is prohibited, and GETMAIN (STORAGE?) for even 64-bit addressable storage will not return storage in that area. But why? What harm would befall if it did? A GETMAIN request can specify whether the storage is to be backed by real storage 16 M, 2 G, or anywhere. Why should the application even care? Doesn't virtual storage appear the same regardless where it's physically backed? -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Mon, 21 Apr 2014 11:44:01 -0500, Paul Gilmartin wrote: On Mon, 21 Apr 2014 07:01:50 -0500, Tom Marchant wrote: Real storage with real addresses 2 GiB are below the bar. Real storage with real addresses 2 GiB are above the bar. ITYM = 2 GiB Right. But what do you call real storage with a real address = 3 GiB or thereabouts? Nothing special. The real address is above 2 GiB. I understand that for historic reasons, pertaining only to z/OS, not other OSes, 2 GiB = virtual address 4GiB is prohibited, Prohibited? I think that is too strong a word. IARV64 would not return storage with a virtual address in that range. and GETMAIN (STORAGE?) for even 64-bit addressable storage The service for obtaining storage above the bar is IARV64. will not return storage in that area. Would not return storage in that virtual range. The real storage backing a request could be in that range. Now that range and more (to 32 GiB) is reserved for the use of Java. And there is an undocumented parameter on IARV64, USE2GTO32G, to allow storage to be returned in that virtual storage range. But why? The excuse that I heard was that it was to avoid confusion. Something about the chance That a program with a 64-bit address that used it improperly, clearing bit 32, would incorrectly address the wrong location. My opinion is that it has caused far more confusion. I have seen code that examined an address, and if it found that some of bits 0-31 were non-zero, while bit 32 was one, it declared the address to be invalid. What harm would befall if it did? None, IMO. And we wouldn't have people who believe to this day that the bar is 2 gig think. A GETMAIN request can specify whether the storage is to be backed by real storage 16 M, 2 G, or anywhere. Why should the application even care? Most don't. There are a small number of reasons for having real storage located below the line or below the bar. Actually, when LOC is specified, the real storage to back it is obtained from anywhere until the storage is page-fixed, then LOC is honored. Doesn't virtual storage appear the same regardless where it's physically backed? Yes. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Mon, 21 Apr 2014 11:44:01 -0500, Paul Gilmartin paulgboul...@aim.com wrote: On Mon, 21 Apr 2014 07:01:50 -0500, Tom Marchant wrote: On Fri, 18 Apr 2014 14:07:29 -0500, Paul Gilmartin wrote: Are there separate pools of real storage for above the bar and below the bar? Pools? No. Pools are a software concept. Real storage with real addresses 2 GiB are below the bar. Real storage with real addresses 2 GiB are above the bar. ITYM = 2 GiB But what do you call real storage with a real address = 3 GiB or thereabouts? I understand that for historic reasons, pertaining only to z/OS, not other OSes, 2 GiB = virtual address 4GiB is prohibited, and GETMAIN (STORAGE?) for even 64-bit addressable storage will not return storage in that area. But why? What harm would befall if it did? Search the archives. That hasn't been true for quite some time now. 64-bit Java used it first, but it isn't restricted to Java. I don't know if anything else is using it at this point. The harm / fear was misinterpreting 31-bit addresses as 64-bit addresses in 64-bit programs processing 32-bit address words that had the high order bit set on either unintentionally (leftover garbage) or intentionally for something meaningful. Mark -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS ITIL v3 Foundation Certified mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://search390.techtarget.com/ateExperts/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
I think the term pointer compression is relevant here - to java heap just above 2GB. Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: Mark Zelden m...@mzelden.com To: IBM-MAIN@listserv.ua.edu Date: 21/04/2014 19:19 Subject:Re: SORT ando MEMLIMIT best practice Sent by:IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu On Mon, 21 Apr 2014 11:44:01 -0500, Paul Gilmartin paulgboul...@aim.com wrote: On Mon, 21 Apr 2014 07:01:50 -0500, Tom Marchant wrote: On Fri, 18 Apr 2014 14:07:29 -0500, Paul Gilmartin wrote: Are there separate pools of real storage for above the bar and below the bar? Pools? No. Pools are a software concept. Real storage with real addresses 2 GiB are below the bar. Real storage with real addresses 2 GiB are above the bar. ITYM = 2 GiB But what do you call real storage with a real address = 3 GiB or thereabouts? I understand that for historic reasons, pertaining only to z/OS, not other OSes, 2 GiB = virtual address 4GiB is prohibited, and GETMAIN (STORAGE?) for even 64-bit addressable storage will not return storage in that area. But why? What harm would befall if it did? Search the archives. That hasn't been true for quite some time now. 64-bit Java used it first, but it isn't restricted to Java. I don't know if anything else is using it at this point. The harm / fear was misinterpreting 31-bit addresses as 64-bit addresses in 64-bit programs processing 32-bit address words that had the high order bit set on either unintentionally (leftover garbage) or intentionally for something meaningful. Mark -- Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS ITIL v3 Foundation Certified mailto:m...@mzelden.com Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html Systems Programming expert at http://search390.techtarget.com/ateExperts/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
Except 1MB (and a fortiori 2GB) page frames were designed to do something else: Give better TLB effectiveness - fewer entries to cover the same amount of memory. Cheers, Martin Martin Packer, zChampion, Principal Systems Investigator, Worldwide Banking Center of Excellence, IBM +44-7802-245-584 email: martin_pac...@uk.ibm.com Twitter / Facebook IDs: MartinPacker Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker From: DASDBILL2 dasdbi...@comcast.net To: IBM-MAIN@listserv.ua.edu Date: 18/04/2014 20:26 Subject:Re: SORT ando MEMLIMIT best practice Sent by:IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu To the best of my knowledge, the answer to all your questions except the last one is No. There are separate limits for above and below the bar storage at the address space level, but I don't know about the total system-wide use. And my answer to your last question is Yes. With the possible exception of IBM's fairly recently added feature in which you can request virtual storage that is backed by real storage at the megabyte level instead of at the 4K byte level. This was designed to reduce paging within storage obtained above the bar. What I was really trying to imply was a rule of thumb by which we should not condemn anything before we have tried and measured it and can prove it better or worse than what we had before. Bill Fairchild - Original Message - From: Paul Gilmartin paulgboul...@aim.com To: IBM-MAIN@LISTSERV.UA.EDU Sent: Friday, April 18, 2014 2:07:29 PM Subject: Re: SORT ando MEMLIMIT best practice On Fri, 18 Apr 2014 17:54:01 +, DASDBILL2 dasdbi...@comcast.net wrote: ... Giving a gazillion bytes above the bar to process X does not necessarily mean that process X will ruin system performance. The gazillion bytes could also have come from below the bar (for some values of gazillion). ... Are there separate pools of real storage for above the bar and below the bar? Are there separate pools of page data sets for above the bar and below the bar? Are there separate limits for total (system-wide) virtual storage in use below the bar and above the bar? Are the costs of resources (page and segment tables and other overhead) for above the bar and below the bar different? Unless the answer to at least one of these (or any similar question) is Yes, effect of giving a gazillion bytes is the same above the bar as below. (Or is that what you were implying.) -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
SORT ando MEMLIMIT best practice
On 2014-04-18, at 03:48, R.S. wrote: W dniu 2014-04-18 01:25, Ed Jaffe pisze: On 4/4/2014 12:47 PM, R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Well, My issue (problem?) is I have MEMLIMIT coded, but it's much more than default 2G. And I noticed that some DFSORT jobs consume considerable amounts of memory causing paging. From the other hand I don't want to be stingy, so I'm looking for some recommendations. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. o What are the consequences of allocating SORTWKn to VIO? Many years ago (no longer), I knew the Cooley-Tukey fast Fourier transform algorithm well enough that I could code it from memory. At one point it makes a pass that toucnes the first location in each page in sequence; then the second location in each page; then the third; ... . This is brutal if WSS doesn't fit in real storage. Has anyone optimized FFT to optimize LoR, perhaps rearranging the data on each pass, perhaps even employing PS data sets rather than virual storage? -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
-Original Message- From: IBM Mainframe Discussion List On Behalf Of Paul Gilmartin On 2014-04-18, at 03:48, R.S. wrote: W dniu 2014-04-18 01:25, Ed Jaffe pisze: On 4/4/2014 12:47 PM, R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. (Pssst Hey, Mac -- wanna buy a toll bridge?) AIGF -jc- ** Information contained in this e-mail message and in any attachments thereto is confidential. If you are not the intended recipient, please destroy this message, delete any copies held on your systems, notify the sender immediately, and refrain from using or disclosing all or any part of its content to any other person. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
Paul Gilmartin wrote: R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Agreed. But, as John Gilmore said, IBM's defaults are 'minimal troublesome' [ for 'most installations' -- my own words ]. I usually find that these defaults are Ok and tailoring is reserved for those strange needs. Oh, my MEMLIMIT is NOLIMIT after several attempts to satisfy my DB2 needs and my z/OS Team handling of paging. I'm not using USI exit much these days. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? Yes, I find that DFSORT looks what it can grab: memory, VIO, SORTWKxx and SORTIN parameters for storage usage. Then DFSORT looks at what size those SORT inputs are. At last, it tries to grab whatever it can get to start sorting at all. o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. Use SORTIN parameters to avoid SORTWKxx. Just don't code SORTWKnn in your JCL DD statements. But, I agree, be careful. For myself, I find using SORTWKnn, large REGION and large SORTIN parameters are 'better' for really big sort work. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
DFSORT does look at available resources before it grabs whatever it can. There are DFSORT installation defaults to control how much of the available storage DFSORT will use but I agree the shipped defaults can cause issues in some environments. We put a lot of guidance on these installation defaults in our DFSORT Tuning Guide and in V2R1 we made some changes to try and be a bit less agressive in how we allocate available storage. Have a nice day, Dave Betten DFSMS Performance Engineer IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on 04/18/2014 01:15:51 PM: From: Elardus Engelbrecht elardus.engelbre...@sita.co.za To: IBM-MAIN@listserv.ua.edu, Date: 04/18/2014 01:16 PM Subject: Re: SORT ando MEMLIMIT best practice Sent by: IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu Paul Gilmartin wrote: R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Agreed. But, as John Gilmore said, IBM's defaults are 'minimal troublesome' [ for 'most installations' -- my own words ]. I usually find that these defaults are Ok and tailoring is reserved for those strange needs. Oh, my MEMLIMIT is NOLIMIT after several attempts to satisfy my DB2 needs and my z/OS Team handling of paging. I'm not using USI exit much these days. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? Yes, I find that DFSORT looks what it can grab: memory, VIO, SORTWKxx and SORTIN parameters for storage usage. Then DFSORT looks at what size those SORT inputs are. At last, it tries to grab whatever it can get to start sorting at all. o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. Use SORTIN parameters to avoid SORTWKxx. Just don't code SORTWKnn in your JCL DD statements. But, I agree, be careful. For myself, I find using SORTWKnn, large REGION and large SORTIN parameters are 'better' for really big sort work. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
I don't wish to start a SORT war. :) Syncsort has the GDSM STC which, as I understand it, keeps a history of sort performance/requirments and helps Syncsort choose resource options. I actually have no idea how effective it is for us or others, Does DFSORT have a similar sort history function? -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Betten Sent: Friday, April 18, 2014 10:31 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: SORT ando MEMLIMIT best practice DFSORT does look at available resources before it grabs whatever it can. There are DFSORT installation defaults to control how much of the available storage DFSORT will use but I agree the shipped defaults can cause issues in some environments. We put a lot of guidance on these installation defaults in our DFSORT Tuning Guide and in V2R1 we made some changes to try and be a bit less agressive in how we allocate available storage. Have a nice day, Dave Betten DFSMS Performance Engineer IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on 04/18/2014 01:15:51 PM: From: Elardus Engelbrecht elardus.engelbre...@sita.co.za To: IBM-MAIN@listserv.ua.edu, Date: 04/18/2014 01:16 PM Subject: Re: SORT ando MEMLIMIT best practice Sent by: IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu Paul Gilmartin wrote: R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Agreed. But, as John Gilmore said, IBM's defaults are 'minimal troublesome' [ for 'most installations' -- my own words ]. I usually find that these defaults are Ok and tailoring is reserved for those strange needs. Oh, my MEMLIMIT is NOLIMIT after several attempts to satisfy my DB2 needs and my z/OS Team handling of paging. I'm not using USI exit much these days. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? Yes, I find that DFSORT looks what it can grab: memory, VIO, SORTWKxx and SORTIN parameters for storage usage. Then DFSORT looks at what size those SORT inputs are. At last, it tries to grab whatever it can get to start sorting at all. o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. Use SORTIN parameters to avoid SORTWKxx. Just don't code SORTWKnn in your JCL DD statements. But, I agree, be careful. For myself, I find using SORTWKnn, large REGION and large SORTIN parameters are 'better' for really big sort work. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
DFSORT does not have a sort history function. Have a nice day, Dave Betten DFSMS Performance Engineer IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on 04/18/2014 01:37:02 PM: From: Gibney, Dave gib...@wsu.edu To: IBM-MAIN@listserv.ua.edu, Date: 04/18/2014 01:37 PM Subject: Re: SORT ando MEMLIMIT best practice Sent by: IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu I don't wish to start a SORT war. :) Syncsort has the GDSM STC which, as I understand it, keeps a history of sort performance/requirments and helps Syncsort choose resource options. I actually have no idea how effective it is for usor others, Does DFSORT have a similar sort history function? -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Betten Sent: Friday, April 18, 2014 10:31 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: SORT ando MEMLIMIT best practice DFSORT does look at available resources before it grabs whatever it can. There are DFSORT installation defaults to control how much of theavailable storage DFSORT will use but I agree the shipped defaults can causeissues in some environments. We put a lot of guidance on these installation defaults in our DFSORT Tuning Guide and in V2R1 we made some changes to try and be a bit less agressive in how we allocate available storage. Have a nice day, Dave Betten DFSMS Performance Engineer IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on 04/18/2014 01:15:51 PM: From: Elardus Engelbrecht elardus.engelbre...@sita.co.za To: IBM-MAIN@listserv.ua.edu, Date: 04/18/2014 01:16 PM Subject: Re: SORT ando MEMLIMIT best practice Sent by: IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu Paul Gilmartin wrote: R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Agreed. But, as John Gilmore said, IBM's defaults are 'minimal troublesome' [ for 'most installations' -- my own words ]. I usually find that these defaults are Ok and tailoring is reserved for those strange needs. Oh, my MEMLIMIT is NOLIMIT after several attempts to satisfy my DB2 needs and my z/OS Team handling of paging. I'm not using USI exit much these days. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? Yes, I find that DFSORT looks what it can grab: memory, VIO, SORTWKxx and SORTIN parameters for storage usage. Then DFSORT looks at what size those SORT inputs are. At last, it tries to grab whatever it can get to start sorting at all. o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. Use SORTIN parameters to avoid SORTWKxx. Just don't code SORTWKnn in your JCL DD statements. But, I agree, be careful. For myself, I find using SORTWKnn, large REGION and large SORTIN parameters are 'better' for really big sort work. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
Once the I/O is complete and the buffer has been marked as no longer page-fixed while the I/O is active, that I/O buffer will not experience significant paging if it is being accessed frequently. That's how I/O buffers have always behaved since virtual memory operating systems with paging subsystems were first produced, whether the buffer is below 16M, below 31M, or above 31M. Vendors who build products like DFSORT know how to long-term page-fix their buffers, whether the buffers are above or below the bar, if such page fixing seems useful. In other words, just because a large amount of storage is being used above the bar does not necessarily mean that that storage is being frequently paged. But as more pages are long-term-fixed anywhere in the system, then the paging of everything else in the system may increase. Giving a gazillion bytes above the bar to process X does not necessarily mean that process X will ruin system performance. The gazillion bytes could also have come from below the bar (for some values of gazillion). Any process that is suspected of hogging the system needs to be closely monitored because it has been previously determined that it is hogging the system, not simply because it is using some new type of resource that is not yet well understood. Bill Fairchild - Original Message - From: Paul Gilmartin paulgboul...@aim.com To: IBM-MAIN@LISTSERV.UA.EDU Sent: Friday, April 18, 2014 9:31:13 AM Subject: SORT ando MEMLIMIT best practice On 2014-04-18, at 03:48, R.S. wrote: W dniu 2014-04-18 01:25, Ed Jaffe pisze: On 4/4/2014 12:47 PM, R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Well, My issue (problem?) is I have MEMLIMIT coded, but it's much more than default 2G. And I noticed that some DFSORT jobs consume considerable amounts of memory causing paging. From the other hand I don't want to be stingy, so I'm looking for some recommendations. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. o What are the consequences of allocating SORTWKn to VIO? Many years ago (no longer), I knew the Cooley-Tukey fast Fourier transform algorithm well enough that I could code it from memory. At one point it makes a pass that toucnes the first location in each page in sequence; then the second location in each page; then the third; ... . This is brutal if WSS doesn't fit in real storage. Has anyone optimized FFT to optimize LoR, perhaps rearranging the data on each pass, perhaps even employing PS data sets rather than virual storage? -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
On Fri, 18 Apr 2014 17:54:01 +, DASDBILL2 dasdbi...@comcast.net wrote: ... Giving a gazillion bytes above the bar to process X does not necessarily mean that process X will ruin system performance. The gazillion bytes could also have come from below the bar (for some values of gazillion). ... Are there separate pools of real storage for above the bar and below the bar? Are there separate pools of page data sets for above the bar and below the bar? Are there separate limits for total (system-wide) virtual storage in use below the bar and above the bar? Are the costs of resources (page and segment tables and other overhead) for above the bar and below the bar different? Unless the answer to at least one of these (or any similar question) is Yes, effect of giving a gazillion bytes is the same above the bar as below. (Or is that what you were implying.) -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
The gazillions of bytes above the bar is great, but I still have customers who complaint because we run ALL31 or AMODE 31 or RMODE 31. I attribute this to unchanged legacy programs over the years. Scott ford www.identityforge.com from my IPAD On Apr 18, 2014, at 3:07 PM, Paul Gilmartin paulgboul...@aim.com wrote: On Fri, 18 Apr 2014 17:54:01 +, DASDBILL2 dasdbi...@comcast.net wrote: ... Giving a gazillion bytes above the bar to process X does not necessarily mean that process X will ruin system performance. The gazillion bytes could also have come from below the bar (for some values of gazillion). ... Are there separate pools of real storage for above the bar and below the bar? Are there separate pools of page data sets for above the bar and below the bar? Are there separate limits for total (system-wide) virtual storage in use below the bar and above the bar? Are the costs of resources (page and segment tables and other overhead) for above the bar and below the bar different? Unless the answer to at least one of these (or any similar question) is Yes, effect of giving a gazillion bytes is the same above the bar as below. (Or is that what you were implying.) -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
To the best of my knowledge, the answer to all your questions except the last one is No. There are separate limits for above and below the bar storage at the address space level, but I don't know about the total system-wide use. And my answer to your last question is Yes. With the possible exception of IBM's fairly recently added feature in which you can request virtual storage that is backed by real storage at the megabyte level instead of at the 4K byte level. This was designed to reduce paging within storage obtained above the bar. What I was really trying to imply was a rule of thumb by which we should not condemn anything before we have tried and measured it and can prove it better or worse than what we had before. Bill Fairchild - Original Message - From: Paul Gilmartin paulgboul...@aim.com To: IBM-MAIN@LISTSERV.UA.EDU Sent: Friday, April 18, 2014 2:07:29 PM Subject: Re: SORT ando MEMLIMIT best practice On Fri, 18 Apr 2014 17:54:01 +, DASDBILL2 dasdbi...@comcast.net wrote: ... Giving a gazillion bytes above the bar to process X does not necessarily mean that process X will ruin system performance. The gazillion bytes could also have come from below the bar (for some values of gazillion). ... Are there separate pools of real storage for above the bar and below the bar? Are there separate pools of page data sets for above the bar and below the bar? Are there separate limits for total (system-wide) virtual storage in use below the bar and above the bar? Are the costs of resources (page and segment tables and other overhead) for above the bar and below the bar different? Unless the answer to at least one of these (or any similar question) is Yes, effect of giving a gazillion bytes is the same above the bar as below. (Or is that what you were implying.) -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
I've had much success tuning DFSort (and SyncSort) with appropriate storage tuning parameters. There are many knobs to turn to provide whatever granularity is needed to resolve issues. - Don Imbriale On Fri, Apr 18, 2014 at 1:31 PM, David Betten bet...@us.ibm.com wrote: DFSORT does look at available resources before it grabs whatever it can. There are DFSORT installation defaults to control how much of the available storage DFSORT will use but I agree the shipped defaults can cause issues in some environments. We put a lot of guidance on these installation defaults in our DFSORT Tuning Guide and in V2R1 we made some changes to try and be a bit less agressive in how we allocate available storage. Have a nice day, Dave Betten DFSMS Performance Engineer IBM Corporation email: bet...@us.ibm.com 1-301-240-3809 DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/ IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on 04/18/2014 01:15:51 PM: From: Elardus Engelbrecht elardus.engelbre...@sita.co.za To: IBM-MAIN@listserv.ua.edu, Date: 04/18/2014 01:16 PM Subject: Re: SORT ando MEMLIMIT best practice Sent by: IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu Paul Gilmartin wrote: R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. Agreed. But, as John Gilmore said, IBM's defaults are 'minimal troublesome' [ for 'most installations' -- my own words ]. I usually find that these defaults are Ok and tailoring is reserved for those strange needs. Oh, my MEMLIMIT is NOLIMIT after several attempts to satisfy my DB2 needs and my z/OS Team handling of paging. I'm not using USI exit much these days. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? Yes, I find that DFSORT looks what it can grab: memory, VIO, SORTWKxx and SORTIN parameters for storage usage. Then DFSORT looks at what size those SORT inputs are. At last, it tries to grab whatever it can get to start sorting at all. o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. Use SORTIN parameters to avoid SORTWKxx. Just don't code SORTWKnn in your JCL DD statements. But, I agree, be careful. For myself, I find using SORTWKnn, large REGION and large SORTIN parameters are 'better' for really big sort work. Groete / Greetings Elardus Engelbrecht -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: SORT ando MEMLIMIT best practice
W dniu 2014-04-18 16:31, Paul Gilmartin pisze: On 2014-04-18, at 03:48, R.S. wrote: W dniu 2014-04-18 01:25, Ed Jaffe pisze: On 4/4/2014 12:47 PM, R.S. wrote: If you specify absolutely nothing about MEMLIMIT anywhere, the system-provided default is 2G so obviously you can't go wrong with that in SMFPRMxx. Right. IBM's provided defaults are always optimal. You're kidding, aren't you? Well, My issue (problem?) is I have MEMLIMIT coded, but it's much more than default 2G. And I noticed that some DFSORT jobs consume considerable amounts of memory causing paging. From the other hand I don't want to be stingy, so I'm looking for some recommendations. o Hmmm... I'd think that any parameterization resulting in significant paging of I/O buffers is counterproductive. Is DFSORT aware of this in its design, and does it attempt to tailor its WSS to fit in real storage? o OTOH, paging I/O is reported to be very good. And 64-bit virtual is enough for most plausible data sets. How about eliminating SORTWKn data sets and performing the entire sort in virtual storage? But this approach must pay careful attention to LoR. I eliminated static SORTWK DDs many years ago and let SORT to choose optimal work datasets if any are needed. Sort in memory sounds fine, but there are tasks to large for that. I observed DFSORT jobs consuming 48GB of memory. And that caused excessive paging and problems with OLTP. While I can (and I do) limit such jobs by adding MEMLIMIT to the jobcard, I cannot preclude any new job and maybe other entities hogging memory. -- Radoslaw Skorupka Lodz, Poland --- Tre tej wiadomoci moe zawiera informacje prawnie chronione Banku przeznaczone wycznie do uytku subowego adresata. Odbiorc moe by jedynie jej adresat z wyczeniem dostpu osób trzecich. Jeeli nie jeste adresatem niniejszej wiadomoci lub pracownikiem upowanionym do jej przekazania adresatowi, informujemy, e jej rozpowszechnianie, kopiowanie, rozprowadzanie lub inne dziaanie o podobnym charakterze jest prawnie zabronione i moe by karalne. Jeeli otrzymae t wiadomo omykowo, prosimy niezwocznie zawiadomi nadawc wysyajc odpowied oraz trwale usun t wiadomo wczajc w to wszelkie jej kopie wydrukowane lub zapisane na dysku. This e-mail may contain legally privileged information of the Bank and is intended solely for business use of the addressee. This e-mail may only be received by the addressee and may not be disclosed to any third parties. If you are not the intended addressee of this e-mail or the employee authorized to forward it to the addressee, be advised that any dissemination, copying, distribution or any other similar activity is legally prohibited and may be punishable. If you received this e-mail by mistake please advise the sender immediately by using the reply facility in your e-mail software and delete permanently this e-mail including any copies of it either printed or saved to hard drive. mBank S.A. z siedzib w Warszawie, ul. Senatorska 18, 00-950 Warszawa, www.mBank.pl, e-mail: kont...@mbank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237, NIP: 526-021-50-88. Wedug stanu na dzie 01.01.2014 r. kapita zakadowy mBanku S.A. (w caoci wpacony) wynosi 168.696.052 zote. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
