JZOS DFSort integration was pursued many years ago :-) http://www.ibm.com/support/knowledgecenter/api/content/nl/en-us/SSYKE2_6.0.0/com.ibm.java.zsecurity.api.60.doc/com.ibm.jzos/com/ibm/jzos/DfSort.html
Kirk Wolf Dovetailed Technologies http://dovetail.com On Sat, Apr 2, 2016 at 5:59 AM, Martin Packer <[email protected]> wrote: > Possibly. > > See here for a discussion of DFSORT and JZOS I wrote a while back: > http://www.redbooks.ibm.com/abstracts/sg247779.html > > Especially chapter 23 and, even more so, 23.5 > > My jumping off point was the notion of fields in JZOS, actually. Whether > the mapping between the grammars is bijective or not I don't know. > > But, yes, DFSORT to do the I/O and much of the processing, choreographed > by JZOS, is certainly an idea worth pursuing. > > NOTE: Not much zIIP in that, if any. > > Cheers, Martin > > Martin Packer, > zChampion, Principal Systems Investigator, > Worldwide Cloud & Systems Performance, IBM > > +44-7802-245-584 > > email: [email protected] > > Twitter / Facebook IDs: MartinPacker > Blog: > https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker > > > > From: David Crayford <[email protected]> > To: [email protected] > Date: 02/04/2016 11:35 > Subject: Re: Microprocessor Optimization Primer > Sent by: IBM Mainframe Discussion List <[email protected]> > > > > On 2/04/2016 5:21 PM, Martin Packer wrote: > > Stray thought, Kirk (and yes made public)... > > > > Maybe JZOS should have a way of mapping com.ibm.jzos.fields to DFSORT > > symbols (perhaps for use with JZOS' sort integration). > > That's an interesting idea. Are you suggesting using DFSORT to do the > I/O for sequential data sets? I wonder what the overhead of the JNI > callbacks would be > and if it would be quicker than RecordReader. That would be very cool! > > > And now back to everyone's (ir)regularly :-) scheduled programming... > > > > > > Cheers, Martin > > > > Martin Packer, > > zChampion, Principal Systems Investigator, > > Worldwide Cloud & Systems Performance, IBM > > > > +44-7802-245-584 > > > > email: [email protected] > > > > Twitter / Facebook IDs: MartinPacker > > Blog: > > https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker > > > > > > > > From: Kirk Wolf <[email protected]> > > To: [email protected] > > Date: 01/04/2016 22:48 > > Subject: Re: Microprocessor Optimization Primer > > Sent by: IBM Mainframe Discussion List <[email protected]> > > > > > > > > I'm curious, Andrew - > > > > Have you looked at the com.ibm.jzos.fields package for field mapping of > > records? > > These can be made to use fancy support in the JVM for inline JIT > > replacement of primitive access with machine instructions. > > > > There is also some tools on the JZOS developerWorks site - the JZOS > > RecordGenerator. This tool will allow you to read Assembler DSECTS or > > Cobol copybooks and generate Java classes that use the runtime classes > > from > > com.ibm.jzos.fields. > > > > See: > > > > https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=2acdb076-7582-45b5-93a5-781f90169bd3 > > > > > > > Regards, > > > > Kirk Wolf > > Dovetailed Technologies > > http://dovetail.com > > > > PS> The JZOS RecordReader (for eligible data sets) uses a JNI library > that > > uses best-practices BSAM techniques for efficiently reading sequential > > data > > sets. The JNI interface is block-level, rather than record-level, > which > > helps a great deal. > > The only part that you couldn't do yourself is to make the JNI library > > zAPP/zIIP eligible. The JZOS JNI libraries *are* zAPP eligible, since > > they are part of the IBM JRE. > > > > > > On Thu, Mar 31, 2016 at 7:13 PM, Andrew Rowley > > <[email protected] > >> wrote: > >> On 31/03/2016 23:34, David Crayford wrote: > >> > >>> Thanks for the info Andrew. We use the default GC options and our > >>> workloads are both long running agents and batch jobs. I really do > want > > to > >>> learn a lot more about how to tune the JVM for each specific workload. > > The > >>> SMF 30 records are useless. How do I find out how much time is spent > in > > JIT > >>> overhead compared to actual processing? What tools do you use to > > profile > >>> your JVM apps? > >>> > >> You probably know more about it than I do. I am pretty new to Java on > >> z/OS, coming from C#. I did some basic GC testing by running the same > > job > >> 10 times and taking the average for different GC options. I tried > >> optthruput because it sounded like what I want for Java batch, but from > >> memory total CPU increased by 2 or 3 times. All the GC options I tried > >> changing either increased CPU or made no change, so I have just been > > using > >> the defaults. > >> > >> There is an option somewhere where you can see the time spent in GC, > the > >> Java guys say that as a ROT if it is >5% try a different GC strategy. > > There > >> are so many knobs you can tweak though you could spend an awfully long > > time > >> measuring their effects and interactions. > >> > >> I do know that the effects of allocation patterns are very different > for > >> GC languages and languages like C/C++. In C++ allocating and freeing > > short > >> lived objects is expensive because the memory needs to be tracked. In > GC > >> languages short lived objects are cheap, it is objects that survive to > > GC > >> that are expensive. This can make it a bit difficult to directly > compare > >> performance because there might be different design decisions. > >> > >> Most of my CPU time seems to be I/O related (processing SMF data). Even > >> removing all the reporting seems to make very little difference to the > > CPU > >> consumed. I'm trying to work out if I can reduce the I/O overhead. > >> > >> I don't know about JIT overhead - I think it is probably insignificant > >> because it should only compile the methods you actually invoke. > >> > >> The FTS1 CPU is running CICS, IMS and that one Java batch job spikes > > past > >>> it. > >>> > >> Without other work for competition anything CPU bound will max out the > >> CPU, so that's not necessarily a negative - it just means that it isn't > >> waiting for other things. E.g. at one site the DBAs greatly increased > >> Adabas buffer space. CPU contention increased massively as virtually > >> nothing was waiting for I/O anymore. Caused a few problems, but it's a > > good > >> problem to have. It just means that if it can only get 5% CPU instead > of > >> 50% it will run ~10 times as long. Total CPU time is a better measure > > than > >> CPU%. > >> > >> > >> > >> -- > >> Andrew Rowley > >> Black Hill Software > >> +61 413 302 386 > >> > >> ---------------------------------------------------------------------- > >> For IBM-MAIN subscribe / signoff / archive access instructions, > >> send email to [email protected] with the message: INFO IBM-MAIN > >> > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to [email protected] with the message: INFO IBM-MAIN > > > > > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to [email protected] with the message: INFO IBM-MAIN > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
