Re: Sort Question [AD - Kind of]

2017-10-02 Thread Elardus Engelbrecht
Blaicher, Christopher Y. wrote:

>Syncsort's MFX sort when using the Global Sort Monitor measures systems 
>resources, both on an instantaneous basis and on a historical basis to judge 
>how much memory each sort should use. 

I wish with a really sore heart that all applications and compiled programs do 
those two tricks automagically and in a jiffy.

... like that windoze operating system which remembers your few last used 
application programs and store it in the memory until the page file(s) gets 
full... You run that application again, those associated programs are already 
in the memory, no need to read in from your hard disk.

On the PCs, these languages, Clarion and Clipper, have background things which 
dynamically optimize your memory and files in use while the MSDOS machines are 
idle... Nice features which proved useful.

I believe on C64 from Commodore does the same thing, it does that infamous 
garbage collection when your toy is idle...

So, something which is used to adjust your own performance while running, yes, 
there is some overhead with those dynamic measurements, but if overall CPU 
usage and I/O usage are fewer, that should be good. 

I know about Linklist, COFVLFxx, XCF structures for RACF and such animals, but 
they are somewhat static ('historical basis'), not dynamic like that SyncSort 
Feature.


>On the other hand if there is paging already happening, you only want the sort 
>to take as much as is absolutely necessary to do the job with relative 
>efficiency.

That is another feature I want to steal Sorry, but is it Friday today? ;-)

Thanks for your notes. Much appreciated.

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: Sort Question [AD - Kind of]

2017-10-02 Thread Blaicher, Christopher Y.
Syncsort's MFX sort when using the Global Sort Monitor measures systems 
resources, both on an instantaneous basis and on a historical basis to judge 
how much memory each sort should use.  If you give a MFX sort a REGION=0M it 
may use 8M or 1.5G depending on what the current memory demands are by other 
processes, and what the historical data indicates will be needed over the life 
of the sort.  MFX automatically does all that for you.

If you have a 10G machine and over 2G of it are free, who cares that a sort 
comes along and uses a gig.  It isn't hurting anything.  On the other hand if 
there is paging already happening, you only want the sort to take as much as is 
absolutely necessary to do the job with relative efficiency.

Chris Blaicher
Technical Architect
Mainframe Development
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com

Syncsort Incorporated
2 Blue Hill Plaza #1563
Pearl River, NY 10965
www.syncsort.com

Data quality leader Trillium Software is now a part of Syncsort.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Farley, Peter x23353
Sent: Monday, October 2, 2017 12:49 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Paul Gilmartin
> Sent: Monday, October 02, 2017 12:24 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
>
> On 2017-10-02, at 09:29, Farley, Peter x23353 wrote:
>
> > With respect to the REGION size, remember that SORT (both IBM and its 
> > competitors) is most efficient the more memory you can let him have.  I 
> > have found this especially true for COBOL-based SORT's.
> >
> But does this contend adversely with concurrent jobs?

IMHO it is the job of the Capacity and Performance team to tell the application 
programming teams when application jobs are contending for memory and causing 
paging, especially if it causes SLA's to be missed or near-missed.

> > Consider using a region of at least 500M for large-volume sorts, and be 
> > sure to tell SORT he can use (most of) it via the SORT parameters.  The 
> > more memory he has available the fewer SORTWK's and SORTWK I/O's he will 
> > need to use.
> >
> REGION=0K?

Only if allowed by IEFUSI.  Many large shops do not allow it except with 
special dispensation and approval, specifically because of the issue you raised 
above about memory contention.

> > Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) 
> > or AMP='BUFND=,BUFNI=,RMODE31=ALL'  (VSAM) parameters (as 
> > appropriate).  Consider using SUBSYS=BLSR or SMB parameters for high-volume 
> > VSAM inputs as well.
> >
> I would hope (wish) that DFSORT would supply optimum defaults.  But can an 
> application discern in OPEN exit whether the options were supplied in JCL or 
> as OS defaults?
>
> Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
> era of oscillating merge it relied on EXCP.

Of course SORT uses his own EXCP (or OCO media manager interfaces) where 
possible, as it is far more efficient than QSAM.  I was referring to the OP's 
original COBOL source for this SORT question.  My reply concerned 
application-read input files, not directly-read-by-SORT files.  For JCL sorts I 
always let SORT decide how to optimize his files, but COBOL SORT's act as E15 
and/or E35 exits, so COBOL I/O can and often is being used for those I/O's, 
whether directly for the file to be sorted or a multiplicity of inputs used to 
construct the SORT input and output records.

> > Memory is (relatively) inexpensive, so use it to your advantage.  Your 
> > SLA's will appreciate it.

Peter
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN





ATTENTION: -

The information contained in this message (including any files transmitted with 
this message) may contain proprietary, trade secret or other confidential 
and/or legally privileged information. Any pricing information contained in 
this message or in any files transmitted with this message is always 
confidential and cannot be shared with any third parties without prior written 
approval from Syncsort. This message is intended to be read only by the 
individual