Re: Why sort (was Microprocessor Optimization Primer)

[Paul Gilmartin]

On 2016-04-03, at 19:25, David Betten wrote:

> First of all, full disclaimer that I was in DFSORT development for about 8
> years so I might be biased.  But I just want to share a few thoughts.
> 
> First the idea of loading all the data into a large hashmap to do the sort
> tends to eliminate one very important thing and that's overlap.
> Essentially, you read the entire input, conduct your massive hashsort, and
> then write the output with no overlap of those three phases.  ...
>  
Strawman.  Or red herring.  Or some metaphor.

You seem to have deliberately made an adverse choice so you can refute
it.  Rather than hash, use a B-tree so sorting fully overlaps input.

One might argue that given sufficient page data space any sort could be
performed in virtual storage.  I suspect performance would be suboptimal.

I suspect that for a large enough data set Cooley-Tookey FFT brutally
defies LoR.  But some of the operations in C-T are hauntingly similar
to a balanced merge.  Might sorting techniques with workfiles implement
a C-T that outperforms a virtual storage implementation?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: RLSE does not work with PGM=FTP

[Paul Gilmartin]

On Sun, 3 Apr 2016 21:11:59 -0500, John McKown wrote:
>
>​Dr. Merrill,
>I know that I'm not in your league. But it looks to me like what you're
>doing is allocating a DSN using JCL, but directing the FTP process to write
>to it using dynamic allocation. So you never really open the DD INFILE or
>do a CLOSE on it. IIRC, RLSE is done by the CLOSE operation. Shouldn't your
>GET look more like:
>
>GET CICS2.terse +
>//DD:INFILE​ (replace
> 
And, perhaps likewise irritating, I've discovered that overriding DCB 
attributes in
the DD statement are ineffective.  FTP will use those in the DSCB regardless.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Ed Jaffe]

On 4/3/2016 6:21 PM, Ed Jaffe wrote:
DFSORT, Syncsort, etc. use the CPC/UPT hardware instructions to 
implement the fastest sort on the platform.


Typo. Of course, I meant to write CFC/UPT... :-[

--
Edward E Jaffe
Phoenix Software International, Inc
831 Parkview Drive North
El Segundo, CA 90245
http://www.phoenixsoftware.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: RLSE does not work with PGM=FTP

[Tom Marchant]

On Sun, 3 Apr 2016 20:29:09 -0500, Barry Merrill wrote:

>Using IBM FTP CS V2R1, the uploaded INFILE's excess space is not RLSE'd.
>
>//FTPUP EXEC PGM=FTP,PARM='(EXIT=4'
>//SYSPRINT DD SYSOUT=*,DCB=BLKSIZE=133
>//SYSABEND DD SYSOUT=*
>//SYSOUT   DD SYSOUT=*
>//FTPOUT   DD SYSOUT=*
>//INFILE   DD DSN=MXGLRG.CICS.UNTERSE,DISP=(NEW,CATLG,CATLG),
>//UNIT=(3390,1),DSNTYPE=LARGE,
>//DSORG=PS,RECFM=FB,LRECL=1024,BLKSIZE=6144,
>//SPACE=(CYL,(500,500),RLSE)
>//SYSINDD *
>99.99.99.99
>USERID   PASSWORDPHRASE
>BINARY
>GET CICS2.terse +
>  'MXGLRG.CICS.UNTERSE' (replace
>CLOSE
>QUIT

RLSE is not an attribute that stays with the data set. AFAIK, it only applies 
to the access with that DD statement. If you were to run a step like this:

//COPY   EXEC PGM=IEBGENER
//SYSPRINT  DD  SYSOUT=*
//ALLOC DD  DISP=(NEW,CTLG),DSN=SOME.DATA.SET,
//  SPACE=(TRK,(20,20),RLSE)
//SYSUT1DD  *
record 1
record 2
//SYSUT2DD  DISP=OLD,DSN=SOME.DATA.SET

I think that you would find that the excess space for SOME.DATA.SET is not 
released because the data is not written using the DD statement that has 
RLSE specified.

Likewise, FTP isn't using the INFILE DD statement.

This is only a guess, based upon my understanding of how RLSE works. 
I could be wrong.

-- 
Tom Marchant

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[John McKown]

[image: Mic Drop]
On Apr 3, 2016 20:53, "David Crayford"  wrote:
>
> On 4/04/2016 7:41 AM, John McKown wrote:
>>
>>
>> ​I'm not an application programmer. But I can just imagine the looks of
>> astonishment and the "talk", if I were to write a COBOL program which
does
>> a SORT verb with INPUT PROCEDURE IS and OUTPUT PROCEDURE IS which only
did
>> a SORT FIELDS=COPY operation. Even more astonishment if I coded the
INCLUDE
>> or EXCLUDE to subset my data in addition to, or instead of, using COBOL
>> code. I don't know if such coding would pass the majority of the "peer
>> review" type processes. I'd love to try. Especially if I were smart
enough
>> to do so initially and keep the output listing. Then allow code review to
>> force me to use normal COBOL methods. And then show the differences,
>> assuming the SORT method actually is superior. Of course, I'd better know
>> my management. I was at one shop (sysprog) where my boss (sysprog +
>> manager) did that with a major application that would max the 3083 (long
>> ago). Basically he proved it was due a flawed design. Unfortunately, that
>> cost him him his job because the design was actually done by the head of
>> the company (software development company).
>>
>
> I'm sure the application folks would thinks you're a crazy, performance
obsessed systems programmer and should go back to your cave!

​And they'd be right! And they do, sometimes. But, my management would
adore it __IF IT COULD BE DONE RELIABLY BY THE REGULAR PROGRAMMERS__. Why?
Because more than __anything__ else at present, they want to decrease the
cost of I.T. (They consider it a "money pit" and seem to emotionally
consider it to be an "unnecessary" expense which is not really related to
the core business) . So if a technique, if consistently applied, would
allow them to reduce the MSU cap, thus reducing our software bill, they
want it to be done.​ I was typing more, but really got way too sarcastic.

> FileManager was developed at the IBM APC labs in Perth. I worked with one
of the lead developers on that product and they try to utilize DFSORT as
much as possible.

> There must be significant man years of work optimizing the I/O in DFSORT.
It's sensible to try and leverage that. In the case of Andrews I/O bound
product he could possibly
> significantly accelerate the throughput if he could somehow hook into
sort. Is it a big deal that DFSORT doesn't run on a zIIP when most of the
workload is I/O bound?
>
>
http://www.ibm.com/support/knowledgecenter/SSXJAV_13.1.0/com.ibm.filemanager.doc_13.1/base/funtips.htm
>
> LOL! IBM had to write a FASTREXX subset because standard REXX was a dog!
>
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: RLSE does not work with PGM=FTP

[John McKown]

On Sun, Apr 3, 2016 at 8:29 PM, Barry Merrill  wrote:

> Using IBM FTP CS V2R1, the uploaded INFILE's excess space is not RLSE'd.
>
>
>
> //FTPUP EXEC PGM=FTP,PARM='(EXIT=4'
> //SYSPRINT DD SYSOUT=*,DCB=BLKSIZE=133
> //SYSABEND DD SYSOUT=*
> //SYSOUT   DD SYSOUT=*
> //FTPOUT   DD SYSOUT=*
> //INFILE   DD DSN=MXGLRG.CICS.UNTERSE,DISP=(NEW,CATLG,CATLG),
> //UNIT=(3390,1),DSNTYPE=LARGE,
> //DSORG=PS,RECFM=FB,LRECL=1024,BLKSIZE=6144,
> //SPACE=(CYL,(500,500),RLSE)
> //SYSINDD *
> 99.99.99.99
> USERID   PASSWORDPHRASE
> BINARY
> GET CICS2.terse +
>   'MXGLRG.CICS.UNTERSE' (replace
> CLOSE
> QUIT
>
>
> I found one reference that CLOSE TYPE=T prevents RLSE, but no reference
>
> if FTP uses TYPE=T.
>

​Dr. Merrill,
I know that I'm not in your league. But it looks to me like what you're
doing is allocating a DSN using JCL, but directing the FTP process to write
to it using dynamic allocation. So you never really open the DD INFILE or
do a CLOSE on it. IIRC, RLSE is done by the CLOSE operation. Shouldn't your
GET look more like:

GET CICS2.terse +
//DD:INFILE​ (replace

ref:
https://www.ibm.com/support/knowledgecenter/SSLTBW_1.13.0/com.ibm.zos.r13.halu001/dd_name_support.htm%23dd_name_support?lang=en



> Barry
>
> Herbert W. "Barry" Merrill, PhD
>
> President-Programmer
>
> MXG Software
>
> Merrill Consultants
>
> 10717 Cromwell Drive
>
> Dallas, TX 75229-5112
>
> ba...@mxg.com 
>
> Fax:  214 350 3694 - Still works, received as email
>
> Tel:  214 351 1966 - Unreliable, please use email
>
>

-- 
How many surrealists does it take to screw in a lightbulb? One to hold the
giraffe and one to fill the bathtub with brightly colored power tools.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[David Crayford]

On 4/04/2016 7:41 AM, John McKown wrote:

On Sun, Apr 3, 2016 at 6:00 PM, Andrew Rowley 
wrote:


On 3/04/2016 22:43, David Crayford wrote:


Good question! Sort can be utilised for other purposes than sorting, it
can be used as an I/O engine. DFSORT (or Syncsort) implements bespoke
highly optinized I/O using low-level programming interfaces such as chained
EXCPs which are significantly faster than using standard access methods
like QSAM or BSAM, including overlapping BSAM I/O. DFSORT has exit routines
(callbacks) which get called for each record. Basically it's supercharged
I/O. One of our products does just that as do many others. IIRC, IBM
FileManager uses sort for I/O. The trouble with using this technique with
Java is the JNI/callback overhead.


I'm aware of the efficient I/O, but I'm more interested in the use to put
data into a particular order. My own programs I never sort input data,
frequently sort small subsets of data during processing (likely always too
small quantities for something like DFSORT) and almost always sort for
presentation. Presentation is hopefully also too small quantities for
DFSORT.

It is an interesting idea though to use it to read data via the exits
without actually giving it back to DFSORT to process.


​I'm not an application programmer. But I can just imagine the looks of
astonishment and the "talk", if I were to write a COBOL program which does
a SORT verb with INPUT PROCEDURE IS and OUTPUT PROCEDURE IS which only did
a SORT FIELDS=COPY operation. Even more astonishment if I coded the INCLUDE
or EXCLUDE to subset my data in addition to, or instead of, using COBOL
code. I don't know if such coding would pass the majority of the "peer
review" type processes. I'd love to try. Especially if I were smart enough
to do so initially and keep the output listing. Then allow code review to
force me to use normal COBOL methods. And then show the differences,
assuming the SORT method actually is superior. Of course, I'd better know
my management. I was at one shop (sysprog) where my boss (sysprog +
manager) did that with a major application that would max the 3083 (long
ago). Basically he proved it was due a flawed design. Unfortunately, that
cost him him his job because the design was actually done by the head of
the company (software development company).



I'm sure the application folks would thinks you're a crazy, performance 
obsessed systems programmer and should go back to your cave!


FileManager was developed at the IBM APC labs in Perth. I worked with 
one of the lead developers on that product and they try to utilize 
DFSORT as much as possible.
There must be significant man years of work optimizing the I/O in 
DFSORT. It's sensible to try and leverage that. In the case of Andrews 
I/O bound product he could possibly
significantly accelerate the throughput if he could somehow hook into 
sort. Is it a big deal that DFSORT doesn't run on a zIIP when most of 
the workload is I/O bound?


http://www.ibm.com/support/knowledgecenter/SSXJAV_13.1.0/com.ibm.filemanager.doc_13.1/base/funtips.htm

LOL! IBM had to write a FASTREXX subset because standard REXX was a dog!





--
Andrew Rowley
Black Hill Software
+61 413 302 386

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN






--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Microprocessor Optimization Primer

[Gabe Goldberg]

I'm behind reading this epic thread, but I sent a question related to it 
to Bill Collier, instructor I had at IBM in 1968 while in full-time 
training for OS/360 System Design Department. He runs 
http://www.mpdiag.com and is biographied at 
http://www.bestweb.net/~collier -- so is valuable resource on all-things 
multiprocessor:



Bill -- I shared this interesting document with some friends...

Communities category:Linux on System z Open Source
Ecosystem:Microprocessor ...

https://www.ibm.com/developerworks/community/forums/html/topic?id=5cf34211-c8e6-4747-a8c2-f8ff7379150b


...and for some reason we've gotten bogged down on this bit of
minutia, with one person asserting:

Interesting too was mention of "TEST AND SET", which in the 1960's,
was NOT an atomic instruction, and it caused multiprocessing
serialization problems in MVT/MP65 and from thence after, to be
replaced by "Compare and Swap", which WAS atomic.

But the PDF implied to me that now TS "Test and Set" WAS atomic.
Implied, anyhow.

A rewrite of history? A fix? Did I read it wrong?

---

My recollection is that TS was indeed atomic but wasn't adequate for
evolving requirements so was replaced by CS/CDS. There's plenty more
(too much!) in our discussion thread on TS but I think that snippet
suffices. I'll be grateful for any comments you have.


His interesting reply -- posted with permission, attribution requested; 
he's at coll...@acm.org --:


  No chance of getting a good night's sleep now.  Not with a
question to answer which brings back so many memories.

  The short answer.  Yes, Test and Set was atomic.  The trouble
was that as a tie breaker, it was not effective.  You could get
into Alphonse-Gaston situations where neither processor advanced.

  In order to provide a definitive answer, I tried to look up TS.
I have a 1964 copy of the System/360 Principles of Operation
manual.  The TS instruction is not mentioned.  Apparently it came
later.

  CS fixed the AG problem.  A processor might be repeatedly edged
out in the competition and so delayed indefinitely, but at least
other processors were competing successfully and therefore
advancing.

  Were there hardware problems I am not aware of?  Possibly.  If
you want to pursue the question, I could give you the names of
some other folks who were around at the dawn of time, and you
could ask them.

  Here is the long, interesting answer to your question which
dredges up so much fun stuff from the past.

  Was TS necessary?  Is CS necessary?  You can argue that they
weren't.  The most fascinating program I ever saw was Dijkstra's
1965 paper titled "Solution of a problem in concurrent programming
control."  Using just load, store, test, and branch instructions
(that is, no atomic instructions), the program performed a LOCK
function (that is, it allowed at most one program at a time into
a critical section of code) without the possibility of deadlock
(although livelock was still possible).

  In those days it was thought that multiprocessor machines should
be (what Leslie Lamport called) "sequentially consistent".  On an
SC machine the only answers a program could compute would be the
answers it could also compute on a uniprocessor machine.

  If you think about it for a decade or so, you realize that for a
machine to be SC, it must obey at least the following three rules.

  Rule 1.  Instructions are performed in the order defined by the
program.  No out of order execution can be visible.

  Rule 2.  No scoreboarding.  If processor 1 writes into operand A
at the same time that processor 2 reads from A, the value seen by
processor 2 must be either the old value of A or the new value of
A.  It is forbidden for the value to be some combination of bits
or bytes from the old value and the new value.  Similarly if both
processors write into A at the same time.

  This rule has been called "write atomicity".

  Rule 3.  If there are several copies of operand A scattered
throughout the caches of a system, then when processor 1 changes
the value of A, it must appear as if all copies of A change value
at the same instant.

  This rule has also been called "write atomicity".

  To distinguish the two, let's refer to them as "single copy
write atomicity" (SCWA) and "multiple copy write atomicity"
(MCWA).

  Back in those days we thought that it was obvious that
multiprocessors should be SC.  And that therefore they should
successfully execute programs such as Dijkstra's.

  Then reality intruded.  Engineers found that obeying the rules
greatly reduced performance and so came up with the following
argument:  programmers should not create programs which read and
write the same operand at the same time.  If two processes need to
access a shared operand, they should perform a lock operation
which will allow only one of them to proceed and to access the
operand while forcing the other process to wait for the first
process to finish.

  Today machines routinely violate the Rules.  Therefore, today's
machines 

RLSE does not work with PGM=FTP

[Barry Merrill]

Using IBM FTP CS V2R1, the uploaded INFILE's excess space is not RLSE'd.

 

//FTPUP EXEC PGM=FTP,PARM='(EXIT=4'

//SYSPRINT DD SYSOUT=*,DCB=BLKSIZE=133

//SYSABEND DD SYSOUT=*

//SYSOUT   DD SYSOUT=*

//FTPOUT   DD SYSOUT=*

//INFILE   DD DSN=MXGLRG.CICS.UNTERSE,DISP=(NEW,CATLG,CATLG),

//UNIT=(3390,1),DSNTYPE=LARGE,

//DSORG=PS,RECFM=FB,LRECL=1024,BLKSIZE=6144,

//SPACE=(CYL,(500,500),RLSE)

//SYSINDD *

99.99.99.99

USERID   PASSWORDPHRASE

BINARY

GET CICS2.terse +

  'MXGLRG.CICS.UNTERSE' (replace

CLOSE

QUIT

 

I found one reference that CLOSE TYPE=T prevents RLSE, but no reference

if FTP uses TYPE=T.   

 

 

Barry

 

 

Herbert W. "Barry" Merrill, PhD

President-Programmer

MXG Software

Merrill Consultants

10717 Cromwell Drive

Dallas, TX 75229-5112

ba...@mxg.com  

Fax:  214 350 3694 - Still works, received as email

Tel:  214 351 1966 - Unreliable, please use email

 

www.mxg.com  HomePage: FAQ answers most
questions

ad...@mxg.com    License Forms, Invoice, Payment,
ftp information

supp...@mxg.com  Technical Issues 

MXG-L FREE ListServer  http://www.mxg.com/mxg-l_listserver/

 


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[David Betten]

First of all, full disclaimer that I was in DFSORT development for about 8
years so I might be biased.  But I just want to share a few thoughts.

First the idea of loading all the data into a large hashmap to do the sort
tends to eliminate one very important thing and that's overlap.
Essentially, you read the entire input, conduct your massive hashsort, and
then write the output with no overlap of those three phases.  The approach
I prefer is an iterative process of sorting smaller amounts and writing
them to work files (either on disk or in memory) and then at end of input,
you almost immediately begin the output process of merging those sorted
strings.  This technique is very efficient and I can tell you many z/OS
customers are sorting tens to hundreds of gigabytes of data this way.

Second point I'd like to make also is related to overlap.  Sorting the
files allows downstream process to read them sequentially rather than
random gets from say VSAM or a data base.  When you read or write
sequentially, you have opportunities for I/O overlap along with blocking
and chaining.  So you can be reading the next set of data while your
program is processing the previous set of data.  This results in
considerable elapsed time savings and reduction in I/O overhead since more
data is transferred with each I/O.

And that's just my 2 cents!


Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email:  bet...@us.ibm.com

IBM Mainframe Discussion List  wrote on
04/03/2016 07:28:39 PM:

> From: Andrew Rowley 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 04/03/2016 07:32 PM
> Subject: Re: Why sort (was Microprocessor Optimization Primer)
> Sent by: IBM Mainframe Discussion List 
>
> The reason I like Java on Z so much is I got used to using Hashtable in
> C#, then tried to use Rexx stems to do the same thing. (It was semi
> successful but I always felt like it was very fragile due to the
> potential for unexpected values etc. for the stems.) Then I found Java
> had real hash tables. They make so many different problems so much
easier.
>
> A million 1500 byte entries should be about 1.5 GB I think, and I would
> expect a hashmap to handle it without difficulty as long as the real
> storage was available. But typically a hashtable would hold an object
> with the specific items you're interested in rather than the whole 1500
> byte item.
>
> As for sorting a List of a million 1500 byte items - again I would
> expect Java to do this without difficulty as long as real storage is
> available. Java is actually pretty efficient at this because you're
> actually sorting a list of pointers - you go all over memory to do the
> compares, but should be only shuffling 8MB of data in storage if you
> have a million 64 bit pointers. I regularly test EasySMF (written in C#)
> displaying lists of 1,000,000+ items on the PC. It has column click
> sorting, and it copes just fine with 1,000,000+ lists. Sorting a column
> takes a few seconds at most on a not particularly fast PC.
>
> DFSORT seems to be most useful where you need to sort more data than can
> be processed in storage - but I'm wondering how often that really needs
> to be done. I'm not so interested in utilities and databases calling it
> under the covers - more in applications that require records in a
> particular order. Nor am I saying that's wrong - I'm really just asking
> whether languages like Java provide opportunities to eliminate some
sorting.
>
> On 3/04/2016 22:36, John McKown wrote:
> > ​Sure, but how often do you have a Java HashMap which contains, say, a
> > million entries? Oh, and the entries are not something like an "int",
but
> > more like a C struct where the size of each struct is around 1500
bytes.
> > That would require about 1.5 Terabytes of memory. Not many systems have
> > that much to give you for a single "object".​ And yes, we _do_ sort such
> > monsters. Not often, granted, but we're doing a conversion right now
and
> > the programmer is doing work on claims which go back 10 years! That's a
> > _lot_ of data! And , we don't have _any_ data bases, just VSAM and
> > sequential data sets. I've actually used VSAM to do "sorting", by
inserting
> > records randomly, then reading them back in keyed order. The
performance
> > was horrible. DB2, or other database system, could be used in such as
> > manner to avoid sorting. But I'd bet it would also be horrible. Of
course,
> > if you're reading an already existing VSAM keyed file, or a database,
then
> > you're golden. I'd bet most of the data in the non-z/OS world is kept
in
> > such a manner, as opposed to a regular "file".
> >
> > On z/OS, REXX has "stem" variables which are "content addressable",
much
> > like a HashMap (keep type HaspMap, ). The COBOL language doesn't
have
> > anything like this built in. Neither does PL/I. Of course, IBM's Java
for
> > z/OS does. As do other 

Re: Why sort (was Microprocessor Optimization Primer)

[Ed Jaffe]

On 4/3/2016 4:28 PM, Andrew Rowley wrote:
DFSORT seems to be most useful where you need to sort more data than 
can be processed in storage - but I'm wondering how often that really 
needs to be done. I'm not so interested in utilities and databases 
calling it under the covers - more in applications that require 
records in a particular order. Nor am I saying that's wrong - I'm 
really just asking whether languages like Java provide opportunities 
to eliminate some sorting.


DFSORT, Syncsort, etc. use the CPC/UPT hardware instructions to 
implement the fastest sort on the platform.


Are there java methods that also do this? Or do they use relatively 
inefficient software-based algorithms?


--
Edward E Jaffe
Phoenix Software International, Inc
831 Parkview Drive North
El Segundo, CA 90245
http://www.phoenixsoftware.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: New to Z/OSMF

[Tracy Adams]

Thanks for the reply, it is 2.1.

Sent from my iPhone

> On Apr 3, 2016, at 3:23 PM, Jesse 1 Robinson  wrote:
> 
> You don't say what z/OS you're running. I made it a point to research z/OSMF 
> at SHARE in San Antonio. Learned that z/OS 2.1 really needs UI90034 to make 
> the tool more manageable as in 2.2. I have installed UI90034 but not yet had 
> a chance to dive into upgrade from R13. UI90034 is supposed to make life a 
> lot easier under 2.1.
> 
> .
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler 
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-302-7535 Office
> robin...@sce.com
> 
> 
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Tracy Adams
> Sent: Sunday, April 03, 2016 10:41 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: (External):New to Z/OSMF
> 
> Trying to get this up and running with all plugins.  All steps go well 
> including the -finish step and the step to add the plugins with the 
> overridefile.  I start IZUSVR1 up and wait to start the browser interface.  
> After I see the stc quiet down I launch the browser only to see the core apps 
> and no plugins.  I look at the messages in the stc and see it is using 
> /var/zosmf/configuration and it stores the current values in 
> "active_configuration".  Sure enough, all settings for IZU_xx_CONFIGURE and 
> the the others are all set to N.  The files I used in the setup scripts were 
> all in  the /etc/zosmf directories and my izuconfig1.cfg file there has all 
> the plugins as Y.
> 
> 
> What did I do wrong and how do I get the "active" config file to recognize 
> the plugins?
> 
> 
> TIA,
> 
> 
> Tracy
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[John McKown]

On Sun, Apr 3, 2016 at 6:28 PM, Andrew Rowley 
wrote:

> The reason I like Java on Z so much is I got used to using Hashtable in
> C#, then tried to use Rexx stems to do the same thing. (It was semi
> successful but I always felt like it was very fragile due to the potential
> for unexpected values etc. for the stems.) Then I found Java had real hash
> tables. They make so many different problems so much easier.
>
> A million 1500 byte entries should be about 1.5 GB I think, and I would
> expect a hashmap to handle it without difficulty as long as the real
> storage was available. But typically a hashtable would hold an object with
> the specific items you're interested in rather than the whole 1500 byte
> item.
>

​Yeah, my arithmetic is really bad.​



> As for sorting a List of a million 1500 byte items - again I would expect
> Java to do this without difficulty as long as real storage is available.
> Java is actually pretty efficient at this because you're actually sorting a
> list of pointers - you go all over memory to do the compares


​Hum, just a concern or mine (it may be obsolete), would be the working set
in memory of doing that. z/OS, even on our small shop, is probably running
8 other batch jobs, 5 TSO users (we're small), and 7 CICS regions. I'd
worry about sizing the real memory on the LPAR if all 8 jobs were "going
all over memory". But, again, I have a very small z9BC system, so I worry
about things that the big boy would sneer at.​



> , but should be only shuffling 8MB of data in storage if you have a
> million 64 bit pointers. I regularly test EasySMF (written in C#)
> displaying lists of 1,000,000+ items on the PC. It has column click
> sorting, and it copes just fine with 1,000,000+ lists. Sorting a column
> takes a few seconds at most on a not particularly fast PC.
>
> DFSORT seems to be most useful where you need to sort more data than can
> be processed in storage - but I'm wondering how often that really needs to
> be done. I'm not so interested in utilities and databases calling it under
> the covers - more in applications that require records in a particular
> order. Nor am I saying that's wrong - I'm really just asking whether
> languages like Java provide opportunities to eliminate some sorting.
>

You have a good point about using SORT directly. Let the thing in the
infrastructure use sort, like SQL "ORDER BY" or other things. Of course, it
would be easier in our shop to do this if the COBOL language had a hashing
facility built into it. Most of our code is COBOL and a CA product called
EasyTrieve. We don't have any "fancy" or "up to date" languages like Java,
Python, Ruby, Go, ... insert others ... .

Took me a while to write and I had to rewrite a number of times when my
current bitterness about things at work got to be too much. I'm going to go
watch some ALF and "Get Smart" episodes to cheer up.


>
>
> --
> Andrew Rowley
> Black Hill Software
> +61 413 302 386
>


-- 
How many surrealists does it take to screw in a lightbulb? One to hold the
giraffe and one to fill the bathtub with brightly colored power tools.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[John McKown]

On Sun, Apr 3, 2016 at 6:00 PM, Andrew Rowley 
wrote:

> On 3/04/2016 22:43, David Crayford wrote:
>
>> Good question! Sort can be utilised for other purposes than sorting, it
>> can be used as an I/O engine. DFSORT (or Syncsort) implements bespoke
>> highly optinized I/O using low-level programming interfaces such as chained
>> EXCPs which are significantly faster than using standard access methods
>> like QSAM or BSAM, including overlapping BSAM I/O. DFSORT has exit routines
>> (callbacks) which get called for each record. Basically it's supercharged
>> I/O. One of our products does just that as do many others. IIRC, IBM
>> FileManager uses sort for I/O. The trouble with using this technique with
>> Java is the JNI/callback overhead.
>>
>
> I'm aware of the efficient I/O, but I'm more interested in the use to put
> data into a particular order. My own programs I never sort input data,
> frequently sort small subsets of data during processing (likely always too
> small quantities for something like DFSORT) and almost always sort for
> presentation. Presentation is hopefully also too small quantities for
> DFSORT.
>
> It is an interesting idea though to use it to read data via the exits
> without actually giving it back to DFSORT to process.


​I'm not an application programmer. But I can just imagine the looks of
astonishment and the "talk", if I were to write a COBOL program which does
a SORT verb with INPUT PROCEDURE IS and OUTPUT PROCEDURE IS which only did
a SORT FIELDS=COPY operation. Even more astonishment if I coded the INCLUDE
or EXCLUDE to subset my data in addition to, or instead of, using COBOL
code. I don't know if such coding would pass the majority of the "peer
review" type processes. I'd love to try. Especially if I were smart enough
to do so initially and keep the output listing. Then allow code review to
force me to use normal COBOL methods. And then show the differences,
assuming the SORT method actually is superior. Of course, I'd better know
my management. I was at one shop (sysprog) where my boss (sysprog +
manager) did that with a major application that would max the 3083 (long
ago). Basically he proved it was due a flawed design. Unfortunately, that
cost him him his job because the design was actually done by the head of
the company (software development company).



>
>
> --
> Andrew Rowley
> Black Hill Software
> +61 413 302 386
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>



-- 
How many surrealists does it take to screw in a lightbulb? One to hold the
giraffe and one to fill the bathtub with brightly colored power tools.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Andrew Rowley]

The reason I like Java on Z so much is I got used to using Hashtable in 
C#, then tried to use Rexx stems to do the same thing. (It was semi 
successful but I always felt like it was very fragile due to the 
potential for unexpected values etc. for the stems.) Then I found Java 
had real hash tables. They make so many different problems so much easier.


A million 1500 byte entries should be about 1.5 GB I think, and I would 
expect a hashmap to handle it without difficulty as long as the real 
storage was available. But typically a hashtable would hold an object 
with the specific items you're interested in rather than the whole 1500 
byte item.


As for sorting a List of a million 1500 byte items - again I would 
expect Java to do this without difficulty as long as real storage is 
available. Java is actually pretty efficient at this because you're 
actually sorting a list of pointers - you go all over memory to do the 
compares, but should be only shuffling 8MB of data in storage if you 
have a million 64 bit pointers. I regularly test EasySMF (written in C#) 
displaying lists of 1,000,000+ items on the PC. It has column click 
sorting, and it copes just fine with 1,000,000+ lists. Sorting a column 
takes a few seconds at most on a not particularly fast PC.


DFSORT seems to be most useful where you need to sort more data than can 
be processed in storage - but I'm wondering how often that really needs 
to be done. I'm not so interested in utilities and databases calling it 
under the covers - more in applications that require records in a 
particular order. Nor am I saying that's wrong - I'm really just asking 
whether languages like Java provide opportunities to eliminate some sorting.


On 3/04/2016 22:36, John McKown wrote:

​Sure, but how often do you have a Java HashMap which contains, say, a
million entries? Oh, and the entries are not something like an "int", but
more like a C struct where the size of each struct is around 1500 bytes.
That would require about 1.5 Terabytes of memory. Not many systems have
that much to give you for a single "object".​ And yes, we _do_ sort such
monsters. Not often, granted, but we're doing a conversion right now and
the programmer is doing work on claims which go back 10 years! That's a
_lot_ of data! And , we don't have _any_ data bases, just VSAM and
sequential data sets. I've actually used VSAM to do "sorting", by inserting
records randomly, then reading them back in keyed order. The performance
was horrible. DB2, or other database system, could be used in such as
manner to avoid sorting. But I'd bet it would also be horrible. Of course,
if you're reading an already existing VSAM keyed file, or a database, then
you're golden. I'd bet most of the data in the non-z/OS world is kept in
such a manner, as opposed to a regular "file".

On z/OS, REXX has "stem" variables which are "content addressable", much
like a HashMap (keep type HaspMap, ). The COBOL language doesn't have
anything like this built in. Neither does PL/I. Of course, IBM's Java for
z/OS does. As do other languages in the UNIX environment such as Perl. But
there just aren't as many of them in z/OS due to the effort to make them
work in an EBCDIC environment instead of an ASCII (or Unicode) environment.
For Perl, Larry Wall just said "forget it, we're not doing it any more". I
know that there is a port of LUA ( http://lua4z.com/ ), but I don't know
how popular it is. Unfortunately, z/OS people (programmers, sysprogs, and
management) don't really seem to be very interested in doing UNIX type work
on z/OS. Possibly because "it's too expensive!" or "it's not how we have
done things in the past and it's too difficult to bother learning." Or,
maybe, just plain NIH syndrome (Not Invented Here). I mean, have you read
the screams here about the latest COBOL requiring PDSEs for their
executable output? You'd think that they'd been told to convert their COBOL
to FORTRAN.





--
Andrew Rowley
Black Hill Software
+61 413 302 386

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Andrew Rowley]

On 3/04/2016 22:43, David Crayford wrote:
Good question! Sort can be utilised for other purposes than sorting, 
it can be used as an I/O engine. DFSORT (or Syncsort) implements 
bespoke highly optinized I/O using low-level programming interfaces 
such as chained EXCPs which are significantly faster than using 
standard access methods like QSAM or BSAM, including overlapping BSAM 
I/O. DFSORT has exit routines (callbacks) which get called for each 
record. Basically it's supercharged I/O. One of our products does just 
that as do many others. IIRC, IBM FileManager uses sort for I/O. The 
trouble with using this technique with Java is the JNI/callback overhead.


I'm aware of the efficient I/O, but I'm more interested in the use to 
put data into a particular order. My own programs I never sort input 
data, frequently sort small subsets of data during processing (likely 
always too small quantities for something like DFSORT) and almost always 
sort for presentation. Presentation is hopefully also too small 
quantities for DFSORT.


It is an interesting idea though to use it to read data via the exits 
without actually giving it back to DFSORT to process.


--
Andrew Rowley
Black Hill Software
+61 413 302 386

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Redpiece on SDSFAUX

[John McKown]

Learn,Adopt, Deploy. For those wondering about SDSFAUX.

 http://www.redbooks.ibm.com/abstracts/redp5337.html
-- 
How many surrealists does it take to screw in a lightbulb? One to hold the
giraffe and one to fill the bathtub with brightly colored power tools.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Tony Harminc]

On 3 April 2016 at 02:50, Andrew Rowley  wrote:
> One question that puzzles me (maybe it's my lack of an application
> programming background): Why is sort used so much on z/OS?

As others have pointed out, sort on z/OS (whether IBM's or other
vendors') can be used for all sorts (heh) of general I/O with high
performance. But "sort" also covers the notion of merge, and more
generally of collation. Many languages have constructs that implicitly
sort, and all relational (and probably other) databases will sort
implicitly as required, whether they implement their own sorts, or
call the system one. The database product I worked on 20 years ago had
three levels of sort: for a few rows it did its own in-storage sort,
for thousands of rows it did it's own with work files, and for bigger
stuff it called the system sort. Today those thresholds would be x10
at least because of much cheaper and bigger main storage, but the
concept holds.

A historical reason for use of sort on z/OS may be that "way back in
the days of steam powered computing", main storage was very expensive,
disk was expensive, and tape was cheap. It was not unusual for sort to
use tapes for work files; how else would you sort tens of millions of
records on a machine with, say, 512 kB of storage and a few hundred
megabytes of disk?

UNIX and indeed most other systems didn't start out doing commercial
data processing, and to this day don't do batch processing very well.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Blaicher, Christopher Y.]

I have always argued that a company can buy more CPU, but no one can buy more 
wall clock time.

Yes, sometimes you need CPU to be king, which is why MFX, Syncsort to you old 
timers, has offered multiple optimization options for years.

Chris Blaicher
Technical Architect
Ironstream Development
Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Jesse 1 Robinson
Sent: Sunday, April 03, 2016 4:21 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Why sort (was Microprocessor Optimization Primer)

I used to work for the late great Security Pacific, at the time the largest 
bank based in Los Angeles. When DFSORT was a pimply-faced teenager, some of us 
sysprogs were invited to Santa Teresa to meet with product developers to share 
some real-world feedback. They were a young and earnest bunch. They wanted us 
to help them decide between two frequently conflicting goals:

  Minimize CPU time
  Minimize I/O count

Enhancing one often came at the expense of the other. We couldn't wait to lay 
it on them. Every business day at 2 AM, a messenger would arrive at our data 
center to collect a bag containing all the checks processed that day along with 
reports tied to them. The bag was to be delivered to 'the feds downtown'. If 
the bag was ready for pickup, all was sweetness and light. If the bag was late, 
there would be h*ll to pay. That's all that mattered: wall clock time. It was a 
revelation to the developers.

Every serious business has to sort data for a myriad reasons, all of which boil 
down to this: somewhere along the line--surely more than once--data must be 
processed in some kind of order. Maybe by account number. Maybe by account 
name. Maybe by account value. Each of these needs requires ordering unsorted 
data or data previously sorted for another purpose. Sort is a huge lynchpin in 
the foundation of any large business. Argue about CPU or I/O stats all you 
want. You either meet your 'messenger' deadline or you don't.

.
.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-302-7535 Office
robin...@sce.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Blaicher, Christopher Y.
Sent: Sunday, April 03, 2016 6:52 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Why sort (was Microprocessor Optimization Primer)

Along with the other reasons outlined by others, it significantly improves bulk 
processing, I shy away from the term batch because that has come to have a bad 
connotation.

When dealing with individual transactions, such as an ATM transaction or a web 
transaction, sorted data is not needed.  But, when company goes to process all 
the payments received that day, or checks that cleared, etc., processing is 
much improved when the data coming in is in the same sequence as the existing 
data structure.  It improves because of locality of reference.

Using a relational data base, or any other random access method, doesn't mean 
you have to access it randomly.

Chris Blaicher
Technical Architect
Software Development
Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Andrew Rowley
Sent: Sunday, April 03, 2016 2:51 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Why sort (was Microprocessor Optimization Primer)

On 02/04/2016 10:09 PM, David Crayford wrote:
> IBM switched the magic bit to offload the JZOS JNI C/C++ workload to a
> zIIP so they could do the same for DFSORT. A well engineered library
> could handle the callbacks so the client just reads records like a
> normal API. That would certainly push Java batch up a notch.
One question that puzzles me (maybe it's my lack of an application programming 
background): Why is sort used so much on z/OS?

I know you can then e.g. do grouping based on key changes, but is that really 
necessary in current programs? Is that the reason it is commonly used?

I generally use e.g. Java HashMap, C# Hashtable for grouping so the data 
doesn't need to be sorted. Do other common languages on z/OS provide similar 
functions? (C++ I know.) Are there opportunities to use programming language 
features to avoid sorts altogether?

Andrew Rowley

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN





ATTENTION: -

The information contained in this message (including any files transmitted with 
this message) may contain proprietary, trade secret or other 

Re: Why sort (was Microprocessor Optimization Primer)

[Jesse 1 Robinson]

I used to work for the late great Security Pacific, at the time the largest 
bank based in Los Angeles. When DFSORT was a pimply-faced teenager, some of us 
sysprogs were invited to Santa Teresa to meet with product developers to share 
some real-world feedback. They were a young and earnest bunch. They wanted us 
to help them decide between two frequently conflicting goals:

  Minimize CPU time
  Minimize I/O count

Enhancing one often came at the expense of the other. We couldn't wait to lay 
it on them. Every business day at 2 AM, a messenger would arrive at our data 
center to collect a bag containing all the checks processed that day along with 
reports tied to them. The bag was to be delivered to 'the feds downtown'. If 
the bag was ready for pickup, all was sweetness and light. If the bag was late, 
there would be h*ll to pay. That's all that mattered: wall clock time. It was a 
revelation to the developers. 

Every serious business has to sort data for a myriad reasons, all of which boil 
down to this: somewhere along the line--surely more than once--data must be 
processed in some kind of order. Maybe by account number. Maybe by account 
name. Maybe by account value. Each of these needs requires ordering unsorted 
data or data previously sorted for another purpose. Sort is a huge lynchpin in 
the foundation of any large business. Argue about CPU or I/O stats all you 
want. You either meet your 'messenger' deadline or you don't.  

.
.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler 
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-302-7535 Office
robin...@sce.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Blaicher, Christopher Y.
Sent: Sunday, April 03, 2016 6:52 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Why sort (was Microprocessor Optimization Primer)

Along with the other reasons outlined by others, it significantly improves bulk 
processing, I shy away from the term batch because that has come to have a bad 
connotation.

When dealing with individual transactions, such as an ATM transaction or a web 
transaction, sorted data is not needed.  But, when company goes to process all 
the payments received that day, or checks that cleared, etc., processing is 
much improved when the data coming in is in the same sequence as the existing 
data structure.  It improves because of locality of reference.

Using a relational data base, or any other random access method, doesn't mean 
you have to access it randomly.

Chris Blaicher
Technical Architect
Software Development
Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Andrew Rowley
Sent: Sunday, April 03, 2016 2:51 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Why sort (was Microprocessor Optimization Primer)

On 02/04/2016 10:09 PM, David Crayford wrote:
> IBM switched the magic bit to offload the JZOS JNI C/C++ workload to a 
> zIIP so they could do the same for DFSORT. A well engineered library 
> could handle the callbacks so the client just reads records like a 
> normal API. That would certainly push Java batch up a notch.
One question that puzzles me (maybe it's my lack of an application programming 
background): Why is sort used so much on z/OS?

I know you can then e.g. do grouping based on key changes, but is that really 
necessary in current programs? Is that the reason it is commonly used?

I generally use e.g. Java HashMap, C# Hashtable for grouping so the data 
doesn't need to be sorted. Do other common languages on z/OS provide similar 
functions? (C++ I know.) Are there opportunities to use programming language 
features to avoid sorts altogether?

Andrew Rowley

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: New to Z/OSMF

[Jesse 1 Robinson]

You don't say what z/OS you're running. I made it a point to research z/OSMF at 
SHARE in San Antonio. Learned that z/OS 2.1 really needs UI90034 to make the 
tool more manageable as in 2.2. I have installed UI90034 but not yet had a 
chance to dive into upgrade from R13. UI90034 is supposed to make life a lot 
easier under 2.1.

.
.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler 
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-302-7535 Office
robin...@sce.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tracy Adams
Sent: Sunday, April 03, 2016 10:41 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):New to Z/OSMF

Trying to get this up and running with all plugins.  All steps go well 
including the -finish step and the step to add the plugins with the 
overridefile.  I start IZUSVR1 up and wait to start the browser interface.  
After I see the stc quiet down I launch the browser only to see the core apps 
and no plugins.  I look at the messages in the stc and see it is using 
/var/zosmf/configuration and it stores the current values in 
"active_configuration".  Sure enough, all settings for IZU_xx_CONFIGURE and the 
the others are all set to N.  The files I used in the setup scripts were all in 
 the /etc/zosmf directories and my izuconfig1.cfg file there has all the 
plugins as Y.


What did I do wrong and how do I get the "active" config file to recognize the 
plugins?


TIA,


Tracy

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: TBDISP not displaying entire row

[Dave Salt]

Rather than attempting to display every row in its entirety, the table could 
perhaps just display certain key fields from each row. If a user needs to see 
more detail, they could select a row. This would display a full-screen panel 
(scrollable if required) that displays additional information for the selected 
row.

Dave Salt

SimpList(tm) - try it; you'll get it! 

http://www.mackinney.com/products/program-development/simplist.html  


> Date: Sat, 2 Apr 2016 21:21:44 -0500
> From: eswa...@ca.com
> Subject: TBDISP not displaying entire row
> To: IBM-MAIN@LISTSERV.UA.EDU
> 
> Hi all,
> I have been trying to display all rows of a table using TBDISP (using ISPF 
> table services). the row size can range between 100 to 60k (Table services 
> can allow up to 64K).  while I TBDISP displays all the row  I can't display 
> more than 8 lines per row. which means my rows get truncated. Is this a limit 
> impsed by )MODEL in my PANEL definition or I should be able to display all 
> the rows to their full size.
> If this is truly not possible, is there an alternative? I desperately want to 
> us Table Services to select rows and process them.
> Your help will appreciated greatly.
> 
> Best regards,
> 
> Majeed Eswanil  
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
  
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

New to Z/OSMF

[Tracy Adams]

Trying to get this up and running with all plugins.  All steps go well 
including the -finish step and the step to add the plugins with the 
overridefile.  I start IZUSVR1 up and wait to start the browser interface.  
After I see the stc quiet down I launch the browser only to see the core apps 
and no plugins.  I look at the messages in the stc and see it is using 
/var/zosmf/configuration and it stores the current values in 
"active_configuration".  Sure enough, all settings for IZU_xx_CONFIGURE and the 
the others are all set to N.  The files I used in the setup scripts were all in 
 the /etc/zosmf directories and my izuconfig1.cfg file there has all the 
plugins as Y.


What did I do wrong and how do I get the "active" config file to recognize the 
plugins?


TIA,


Tracy


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[Blaicher, Christopher Y.]

Along with the other reasons outlined by others, it significantly improves bulk 
processing, I shy away from the term batch because that has come to have a bad 
connotation.

When dealing with individual transactions, such as an ATM transaction or a web 
transaction, sorted data is not needed.  But, when company goes to process all 
the payments received that day, or checks that cleared, etc., processing is 
much improved when the data coming in is in the same sequence as the existing 
data structure.  It improves because of locality of reference.

Using a relational data base, or any other random access method, doesn't mean 
you have to access it randomly.

Chris Blaicher
Technical Architect
Software Development
Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Andrew Rowley
Sent: Sunday, April 03, 2016 2:51 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Why sort (was Microprocessor Optimization Primer)

On 02/04/2016 10:09 PM, David Crayford wrote:
> IBM switched the magic bit to offload the JZOS JNI C/C++ workload to a
> zIIP so they could do the same for DFSORT. A well engineered library
> could handle the callbacks so the client just reads records like a
> normal API. That would certainly push Java batch up a notch.
One question that puzzles me (maybe it's my lack of an application programming 
background): Why is sort used so much on z/OS?

I know you can then e.g. do grouping based on key changes, but is that really 
necessary in current programs? Is that the reason it is commonly used?

I generally use e.g. Java HashMap, C# Hashtable for grouping so the data 
doesn't need to be sorted. Do other common languages on z/OS provide similar 
functions? (C++ I know.) Are there opportunities to use programming language 
features to avoid sorts altogether?

Andrew Rowley

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN





ATTENTION: -

The information contained in this message (including any files transmitted with 
this message) may contain proprietary, trade secret or other confidential 
and/or legally privileged information. Any pricing information contained in 
this message or in any files transmitted with this message is always 
confidential and cannot be shared with any third parties without prior written 
approval from Syncsort. This message is intended to be read only by the 
individual or entity to whom it is addressed or by their designee. If the 
reader of this message is not the intended recipient, you are on notice that 
any use, disclosure, copying or distribution of this message, in any form, is 
strictly prohibited. If you have received this message in error, please 
immediately notify the sender and/or Syncsort and destroy all copies of this 
message in your possession, custody or control.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[David Crayford]

On 3/04/2016 2:50 PM, Andrew Rowley wrote:

On 02/04/2016 10:09 PM, David Crayford wrote:
IBM switched the magic bit to offload the JZOS JNI C/C++ workload to 
a zIIP so they could do the same for DFSORT. A well engineered library
could handle the callbacks so the client just reads records like a 
normal API. That would certainly push Java batch up a notch.
One question that puzzles me (maybe it's my lack of an application 
programming background): Why is sort used so much on z/OS?




Good question! Sort can be utilised for other purposes than sorting, it 
can be used as an I/O engine. DFSORT (or Syncsort) implements bespoke 
highly optinized I/O using low-level programming interfaces such as 
chained EXCPs which are significantly faster than using standard access 
methods like QSAM or BSAM, including overlapping BSAM I/O. DFSORT has 
exit routines (callbacks) which get called for each record. Basically 
it's supercharged I/O. One of our products does just that as do many 
others. IIRC, IBM FileManager uses sort for I/O. The trouble with using 
this technique with Java is the JNI/callback overhead.


I know you can then e.g. do grouping based on key changes, but is that 
really necessary in current programs? Is that the reason it is 
commonly used?


I generally use e.g. Java HashMap, C# Hashtable for grouping so the 
data doesn't need to be sorted. Do other common languages on z/OS 
provide similar functions? (C++ I know.) Are there opportunities to 
use programming language features to avoid sorts altogether?


Andrew Rowley

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Why sort (was Microprocessor Optimization Primer)

[John McKown]

On Sun, Apr 3, 2016 at 1:50 AM, Andrew Rowley 
wrote:

> On 02/04/2016 10:09 PM, David Crayford wrote:
>
>> IBM switched the magic bit to offload the JZOS JNI C/C++ workload to a
>> zIIP so they could do the same for DFSORT. A well engineered library
>> could handle the callbacks so the client just reads records like a normal
>> API. That would certainly push Java batch up a notch.
>>
> One question that puzzles me (maybe it's my lack of an application
> programming background): Why is sort used so much on z/OS?
>
> I know you can then e.g. do grouping based on key changes, but is that
> really necessary in current programs? Is that the reason it is commonly
> used?
>

​In my shop, it is used mainly so that the output, such as reports sent to
a web site for perusal or email, are in some order such as ​account number.
Also, DB2 uses it a lot when you do a CREATE INDEX, I think. IDCAMS uses it
when you build an alternate INDEX. This done when a VSAM file (yes, there
are a lot of them still around) is "reorganized" for performance or space
reasons.



>
> I generally use e.g. Java HashMap, C# Hashtable for grouping so the data
> doesn't need to be sorted. Do other common languages on z/OS provide
> similar functions? (C++ I know.) Are there opportunities to use programming
> language features to avoid sorts altogether?
>

​Sure, but how often do you have a Java HashMap which contains, say, a
million entries? Oh, and the entries are not something like an "int", but
more like a C struct where the size of each struct is around 1500 bytes.
That would require about 1.5 Terabytes of memory. Not many systems have
that much to give you for a single "object".​ And yes, we _do_ sort such
monsters. Not often, granted, but we're doing a conversion right now and
the programmer is doing work on claims which go back 10 years! That's a
_lot_ of data! And , we don't have _any_ data bases, just VSAM and
sequential data sets. I've actually used VSAM to do "sorting", by inserting
records randomly, then reading them back in keyed order. The performance
was horrible. DB2, or other database system, could be used in such as
manner to avoid sorting. But I'd bet it would also be horrible. Of course,
if you're reading an already existing VSAM keyed file, or a database, then
you're golden. I'd bet most of the data in the non-z/OS world is kept in
such a manner, as opposed to a regular "file".

On z/OS, REXX has "stem" variables which are "content addressable", much
like a HashMap (keep type HaspMap, ). The COBOL language doesn't have
anything like this built in. Neither does PL/I. Of course, IBM's Java for
z/OS does. As do other languages in the UNIX environment such as Perl. But
there just aren't as many of them in z/OS due to the effort to make them
work in an EBCDIC environment instead of an ASCII (or Unicode) environment.
For Perl, Larry Wall just said "forget it, we're not doing it any more". I
know that there is a port of LUA ( http://lua4z.com/ ), but I don't know
how popular it is. Unfortunately, z/OS people (programmers, sysprogs, and
management) don't really seem to be very interested in doing UNIX type work
on z/OS. Possibly because "it's too expensive!" or "it's not how we have
done things in the past and it's too difficult to bother learning." Or,
maybe, just plain NIH syndrome (Not Invented Here). I mean, have you read
the screams here about the latest COBOL requiring PDSEs for their
executable output? You'd think that they'd been told to convert their COBOL
to FORTRAN.



>
> Andrew Rowley
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>



-- 
How many surrealists does it take to screw in a lightbulb? One to hold the
giraffe and one to fill the bathtub with brightly colored power tools.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Why sort (was Microprocessor Optimization Primer)

[Andrew Rowley]

On 02/04/2016 10:09 PM, David Crayford wrote:
IBM switched the magic bit to offload the JZOS JNI C/C++ workload to a 
zIIP so they could do the same for DFSORT. A well engineered library
could handle the callbacks so the client just reads records like a 
normal API. That would certainly push Java batch up a notch.
One question that puzzles me (maybe it's my lack of an application 
programming background): Why is sort used so much on z/OS?


I know you can then e.g. do grouping based on key changes, but is that 
really necessary in current programs? Is that the reason it is commonly 
used?


I generally use e.g. Java HashMap, C# Hashtable for grouping so the data 
doesn't need to be sorted. Do other common languages on z/OS provide 
similar functions? (C++ I know.) Are there opportunities to use 
programming language features to avoid sorts altogether?


Andrew Rowley

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN