Re: SORT question: Extract varying length string adding suffix after string?

[Farley, Peter x23353]

Thanks a lot Cameron!  SQZ=(VL) did the trick.

BUILD=(1,4,C'PREFIX DATA:',%00,SQA=(VL),C':SUFFIX DATA')

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Cameron Conacher
Sent: Saturday, April 2, 2022 4:53 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT question: Extract varying length string adding suffix after 
string?

If I understand things correctly, you could use Squeeze (SQZ) to eliminate the 
embedded blanks.
Something like SQZ=(SHIFT=LEFT).
Maybe you will need another SORT step to squeeze the entire record?

Thanks,

...Cameron

From: IBM Mainframe Discussion List  On Behalf Of 
Farley, Peter x23353
Sent: Saturday, April 2, 2022 3:45 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] SORT question: Extract varying length string adding suffix 
after string?

I have a use case to extract a varying length string where I need to add both 
prefix and suffix constants to the selected string.

The SORT PARSE operand doesn't seem to do the trick because the FIXLEN 
parameter to PARSE is required and the selected strings will be blank-padded to 
the FIXLEN length.

Is there any way to use SORT for this job or should I be looking at another 
utility? I know I can also do the job using a simple awk script, but I wanted 
to see if SORT could do the job too.

The strings at issue happen to be classic dataset names, PDS.NAME(MBRNAME) 
format. Input and output files are both RECFM=VB.

Example input:

SOME OTHER DATA HERE:DATA.NAME1(MBR001)
SOME OTHER DATA HERE:OTHER.DATA.NAME2(MBR002) 
SOME OTHER DATA HERE:REALLY.LONG.PDS.DATASET.NAME(MBR003)

Sort control cards that I tried:

SORT FIELDS=(26,52,CH,A)
SUM FIELDS=NONE
OUTREC PARSE=(%00=(ABSPOS=26,ENDBEFR=BLANKS,FIXLEN=52)),
BUILD=(1,4,C'PREFIX DATA:',%00,C':SUFFIX DATA')

Output looks like this:

PREFIX DATA:DATA.NAME1(MBR001)  :SUFFIX DATA 
PREFIX DATA:OTHER.DATA.NAME2(MBR002)   
:SUFFIX DATA 
PREFIX DATA:REALLY.LONG.PDS.DATASET.NAME(MBR003)
  :SUFFIX DATA

Desired output would look like this:

PREFIX DATA:DATA.NAME1(MBR001):SUFFIX DATA 
PREFIX DATA:OTHER.DATA.NAME2(MBR002):SUFFIX DATA 
PREFIX DATA:REALLY.LONG.PDS.DATASET.NAME(MBR003):SUFFIX DATA

TIA for any advice you can offer.

Peter
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: Extract varying length string adding suffix after string?

[Cameron Conacher]

If I understand things correctly, you could use Squeeze (SQZ) to eliminate the 
embedded blanks.
Something like SQZ=(SHIFT=LEFT).
Maybe you will need another SORT step to squeeze the entire record?

Thanks,

...Cameron

From: IBM Mainframe Discussion List  On Behalf Of 
Farley, Peter x23353
Sent: Saturday, April 2, 2022 3:45 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] SORT question: Extract varying length string adding suffix 
after string?

I have a use case to extract a varying length string where I need to add both 
prefix and suffix constants to the selected string.

The SORT PARSE operand doesn't seem to do the trick because the FIXLEN 
parameter to PARSE is required and the selected strings will be blank-padded to 
the FIXLEN length.

Is there any way to use SORT for this job or should I be looking at another 
utility? I know I can also do the job using a simple awk script, but I wanted 
to see if SORT could do the job too.

The strings at issue happen to be classic dataset names, 
PDS.NAME<https://isolate.menlosecurity.com/1/3735928037/http:/PDS.NAME>(MBRNAME)
 format. Input and output files are both RECFM=VB.

Example input:

SOME OTHER DATA HERE:DATA.NAME1(MBR001)
SOME OTHER DATA HERE:OTHER.DATA.NAME2(MBR002)
SOME OTHER DATA 
HERE:REALLY.LONG.PDS.DATASET.NAME<https://isolate.menlosecurity.com/1/3735928037/http:/REALLY.LONG.PDS.DATASET.NAME>(MBR003)

Sort control cards that I tried:

SORT FIELDS=(26,52,CH,A)
SUM FIELDS=NONE
OUTREC PARSE=(%00=(ABSPOS=26,ENDBEFR=BLANKS,FIXLEN=52)),
BUILD=(1,4,C'PREFIX DATA:',%00,C':SUFFIX DATA')

Output looks like this:

PREFIX DATA:DATA.NAME1(MBR001) :SUFFIX DATA
PREFIX DATA:OTHER.DATA.NAME2(MBR002) :SUFFIX DATA
PREFIX 
DATA:REALLY.LONG.PDS.DATASET.NAME<https://isolate.menlosecurity.com/1/3735928037/http:/REALLY.LONG.PDS.DATASET.NAME>(MBR003)
 :SUFFIX DATA

Desired output would look like this:

PREFIX DATA:DATA.NAME1(MBR001):SUFFIX DATA
PREFIX DATA:OTHER.DATA.NAME2(MBR002):SUFFIX DATA
PREFIX 
DATA:REALLY.LONG.PDS.DATASET.NAME<https://isolate.menlosecurity.com/1/3735928037/http:/REALLY.LONG.PDS.DATASET.NAME>(MBR003):SUFFIX
 DATA

TIA for any advice you can offer.

Peter
--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with 
the message: INFO IBM-MAIN

American Express made the following annotations

This e-mail was sent to you by a representative of Amex Bank of Canada, P.O. 
Box 3204, Station "F", Toronto, ON, M1W 3W7, www.americanexpress.ca. If you no 
longer wish to receive these e-mails, please notify the sender by reply e-mail.

This e-mail is solely for the intended recipient and may contain confidential 
or privileged information. If you are not the intended recipient, any 
disclosure, copying, use, or distribution of the information included in this 
e-mail is prohibited. If you have received this e-mail in error, please notify 
the sender by reply e-mail and immediately and permanently delete this e-mail 
and any attachments. Thank you.

American Express a fait les remarques suivantes
Ce courriel vous a ?t? envoy? par un repr?sentant de la Banque Amex du Canada, 
C.P. 3204, succursale F, Toronto (Ontario) M1W 3W7, www.americanexpress.ca. Si, 
par la suite, vous ne souhaitez plus recevoir ces courriels, veuillez en aviser 
les exp?diteurs par courriel.

Ce courriel est r?serv? au seul destinataire indiqu? et peut renfermer des 
renseignements confidentiels et privil?gi?s. Si vous n'?tes pas le destinataire 
pr?vu, toute divulgation, duplication, utilisation ou distribution du courriel 
est interdite. Si vous avez re?u ce courriel par erreur, veuillez en aviser 
l'exp?diteur par courriel et d?truire imm?diatement le courriel et toute pi?ce 
jointe. Merci.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SORT question: Extract varying length string adding suffix after string?

[Farley, Peter x23353]

I have a use case to extract a varying length string where I need to add both 
prefix and suffix constants to the selected string.

The SORT PARSE operand doesn't seem to do the trick because the FIXLEN 
parameter to PARSE is required and the selected strings will be blank-padded to 
the FIXLEN length.

Is there any way to use SORT for this job or should I be looking at another 
utility?  I know I can also do the job using a simple awk script, but I wanted 
to see if SORT could do the job too.

The strings at issue happen to be classic dataset names, PDS.NAME(MBRNAME) 
format.  Input and output files are both RECFM=VB.

Example input:

SOME OTHER DATA HERE:DATA.NAME1(MBR001)
SOME OTHER DATA HERE:OTHER.DATA.NAME2(MBR002)
SOME OTHER DATA HERE:REALLY.LONG.PDS.DATASET.NAME(MBR003)

Sort control cards that I tried:

SORT FIELDS=(26,52,CH,A)
 SUM FIELDS=NONE
 OUTREC PARSE=(%00=(ABSPOS=26,ENDBEFR=BLANKS,FIXLEN=52)),
BUILD=(1,4,C'PREFIX DATA:',%00,C':SUFFIX DATA')

Output looks like this:

PREFIX DATA:DATA.NAME1(MBR001)  :SUFFIX DATA
PREFIX DATA:OTHER.DATA.NAME2(MBR002):SUFFIX DATA
PREFIX DATA:REALLY.LONG.PDS.DATASET.NAME(MBR003):SUFFIX DATA

Desired output would look like this:

PREFIX DATA:DATA.NAME1(MBR001):SUFFIX DATA
PREFIX DATA:OTHER.DATA.NAME2(MBR002):SUFFIX DATA
PREFIX DATA:REALLY.LONG.PDS.DATASET.NAME(MBR003):SUFFIX DATA

TIA for any advice you can offer.

Peter
--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Mike Schwab]

Multi step sort least significant to most significant.  Card sorters
did 1 column at a time.
But the single step with correct options is better.

On Thu, Sep 9, 2021 at 7:24 AM Lionel B. Dyck  wrote:
>
> I tried sorting by field 1 ascending and then a second sort with field 2 
> descending (-r) and field1 was no longer in the correct order ☹
>
> But that's what I thought should work as well.
>
>
> Lionel B. Dyck <><
> Website: https://www.lbdsoftware.com
> Github: https://github.com/lbdyck
>
> “Worry more about your character than your reputation. Character is what you 
> are, reputation merely what others think you are.”   - - - John Wooden
>
> -Original Message-
> From: IBM Mainframe Discussion List  On Behalf Of 
> Martin Packer
> Sent: Thursday, September 9, 2021 6:51 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: OMVS Sort Question
>
> Pipe it through two sort commands?
>
> And does sort helpfully avoid rearranging records with the same key value?
>
> Cheers, Martin
>
> Martin Packer
>
> WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: https://mainframeperformancetopics.com
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
> https://anchor.fm/marna-walle
>
> Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
>
>
> From:   "Lionel B. Dyck" 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   09/09/2021 12:25
> Subject:[EXTERNAL] OMVS Sort Question
> Sent by:"IBM Mainframe Discussion List" 
>
>
>
> I know how to use sort - to a limited degree.
>
> This will wort a stem using the 2nd field and sort in reverse:
>
> call bpxwunix 'sort -r -k 2',in.,out.,msg.
>
> What I would like to do is to sort the 2nd field in reverse and then sort the 
> 1st field in ascending - this does not seem possible.
>
> Any tips/pointers?
>
> Thanks
>
>
> Lionel B. Dyck <><
> Website:
> https://www.lbdsoftware.com
>
> Github:
> https://github.com/lbdyck
>
>
> "Worry more about your character than your reputation. Character is what you
> are, reputation merely what others think you are."   - - - John Wooden
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
> lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Lionel B. Dyck]

Thank you - that wasn't clear in the man page.


Lionel B. Dyck <><
Website: https://www.lbdsoftware.com
Github: https://github.com/lbdyck

“Worry more about your character than your reputation. Character is what you 
are, reputation merely what others think you are.”   - - - John Wooden

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: Thursday, September 9, 2021 8:47 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: OMVS Sort Question



> What I would like to do is to sort the 2nd field in reverse and then 
> sort the 1st field in ascending - this does not seem possible.

Lionel,

You need to use the option "r" to perform the descending sort on the 2nd field.

–r  Reverses the order of all comparisons so that sort writes output from 
largest to smallest rather than smallest to largest.

Check this link which explains in detail

https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-sort-start-sort-merge-utility

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Sri h Kolusu]

> What I would like to do is to sort the 2nd field in reverse and then sort
> the 1st field in ascending - this does not seem possible.

Lionel,

You need to use the option "r" to perform the descending sort on the 2nd
field.

–r  Reverses the order of all comparisons so that sort writes output from
largest to smallest rather than smallest to largest.

Check this link which explains in detail

https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-sort-start-sort-merge-utility

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Lionel B. Dyck]

That is brilliant - thank you

This is what I used: call bpxwunix 'sort -k2r -k1 ',in.,out.,msg.


Lionel B. Dyck <><
Website: https://www.lbdsoftware.com
Github: https://github.com/lbdyck

“Worry more about your character than your reputation. Character is what you 
are, reputation merely what others think you are.”   - - - John Wooden

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Patrick Hayward
Sent: Thursday, September 9, 2021 8:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: OMVS Sort Question

Can you not use -k2,2r

Zlinux
>From the man page
KEYDEF  is  F[.C][OPTS][,F[.C][OPTS]]  for start and stop position, where F is 
a field number and C a character position in the field; both are origin 1, and 
the stop position defaults to the
   line's end.  If neither -t nor -b is in effect, characters in a field 
are counted from the beginning of the preceding whitespace. OPTS is one or more 
single-letter ordering options [bdfgiMh‐
   nRrV], which override global ordering options for that key. If no key is 
given, use the entire line as the key.


Unix System Services
You can indicate the start of a sorting key with:

  -k m[.n][options]

  where m and the optional n are positive integers. You can choose options from
  the set bdfiMnr (described previously) to specify the way in which sort does
  comparisons for that sorting key. Ordering options set for a key override 
global
  ordering options. If you do not specify any options for the key, the global
  ordering options are used.


Patrick 

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Patrick Hayward]

Can you not use -k2,2r

Zlinux
From the man page
KEYDEF  is  F[.C][OPTS][,F[.C][OPTS]]  for start and stop position, where F is 
a field number and C a character position in the field; both are origin 1, and 
the stop position defaults to the
   line's end.  If neither -t nor -b is in effect, characters in a field 
are counted from the beginning of the preceding whitespace. OPTS is one or more 
single-letter ordering options [bdfgiMh‐
   nRrV], which override global ordering options for that key. If no key is 
given, use the entire line as the key.


Unix System Services
You can indicate the start of a sorting key with:

  -k m[.n][options]

  where m and the optional n are positive integers. You can choose options from
  the set bdfiMnr (described previously) to specify the way in which sort does
  comparisons for that sorting key. Ordering options set for a key override 
global
  ordering options. If you do not specify any options for the key, the global
  ordering options are used.


Patrick

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Lionel B. Dyck]

I tried sorting by field 1 ascending and then a second sort with field 2 
descending (-r) and field1 was no longer in the correct order ☹

But that's what I thought should work as well.


Lionel B. Dyck <><
Website: https://www.lbdsoftware.com
Github: https://github.com/lbdyck

“Worry more about your character than your reputation. Character is what you 
are, reputation merely what others think you are.”   - - - John Wooden

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Martin Packer
Sent: Thursday, September 9, 2021 6:51 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: OMVS Sort Question

Pipe it through two sort commands?

And does sort helpfully avoid rearranging records with the same key value?

Cheers, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "Lionel B. Dyck" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   09/09/2021 12:25
Subject:    [EXTERNAL] OMVS Sort Question
Sent by:"IBM Mainframe Discussion List" 



I know how to use sort - to a limited degree.

This will wort a stem using the 2nd field and sort in reverse:

call bpxwunix 'sort -r -k 2',in.,out.,msg.

What I would like to do is to sort the 2nd field in reverse and then sort the 
1st field in ascending - this does not seem possible.

Any tips/pointers?

Thanks


Lionel B. Dyck <><
Website: 
https://www.lbdsoftware.com 

Github: 
https://github.com/lbdyck 


"Worry more about your character than your reputation. Character is what you
are, reputation merely what others think you are."   - - - John Wooden

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: OMVS Sort Question

[Martin Packer]

Pipe it through two sort commands?

And does sort helpfully avoid rearranging records with the same key value?

Cheers, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "Lionel B. Dyck" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   09/09/2021 12:25
Subject:[EXTERNAL] OMVS Sort Question
Sent by:"IBM Mainframe Discussion List" 



I know how to use sort - to a limited degree.

This will wort a stem using the 2nd field and sort in reverse:

call bpxwunix 'sort -r -k 2',in.,out.,msg.

What I would like to do is to sort the 2nd field in reverse and then sort
the 1st field in ascending - this does not seem possible.

Any tips/pointers?

Thanks


Lionel B. Dyck <><
Website: 
https://www.lbdsoftware.com 

Github: 
https://github.com/lbdyck 


"Worry more about your character than your reputation. Character is what 
you
are, reputation merely what others think you are."   - - - John Wooden

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

OMVS Sort Question

[Lionel B. Dyck]

I know how to use sort - to a limited degree.

This will wort a stem using the 2nd field and sort in reverse:

call bpxwunix 'sort -r -k 2',in.,out.,msg.

What I would like to do is to sort the 2nd field in reverse and then sort
the 1st field in ascending - this does not seem possible.

Any tips/pointers?

Thanks


Lionel B. Dyck <><
Website: https://www.lbdsoftware.com
Github: https://github.com/lbdyck

"Worry more about your character than your reputation. Character is what you
are, reputation merely what others think you are."   - - - John Wooden

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Batchpipes (was: SORT question: ...)

[Paul Gilmartin]

On Thu, 24 Sep 2020 10:47:37 -0500, Hobart Spitz wrote:

>They find each other via the name of the pipe.
>
Is there an authority administering names of pipes, lest two
users inadvertently choose the same one?  Embedding 
in that name could resolve this.  RACF?  Even so:
//SORTIN  DD DSN=INPUT.PIPE,SUBSYS=PIPE,04
//   DCB=(LRECL=60,RECFM=FB,BLKSIZE=32760)  05

INPUT.PIPE seems too generic.  And must a user intending to submit
multiple job streams concurrently take care to modify those names to
avoid conflicts?

Some of the same concerns apply to POSIX named pipes, mitigated
somewhat by creating the FIFO in the user's $HOME.

Does this example achieve anything that couldn't be as well done with
POSIX FIFOs in the absence of BatchPipes?

Suppose the programmer wants to include customary Pipeline
stages such as SPECS in the pipeline network.  Where would those
be coded?


>On Wed, Sep 23, 2020 at 2:19 PM Martin Packer wrote:
>
>> The pipe name is a data set name.
>>
Could DISP=OLD be used to guarantee serialization?
>>...
>> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Hobart Spitz]

Incorrect.

You use a pipe-fitting on the JCL DD, something like "bpread | sysout q | >
dd=out2 | > dd=out3" for as many outputs as you like.

I'm iffy on the exact syntax for specifying the stream separator, but the
fanout stage can do the same:
   "(end ?) bpread | fo: fanout | sysout q ? fo: | dd=out2 ? fo: | dd=out3"

If you wanted to remove ASA page feeds from, say the last output.
   "(end ?) bpread | fo: fanout | sysout q ? fo: | > dd=out2 ? fo: |
change 1 /1/ / | > dd=out3"

In other words, you have all of CMS/TSO Pipelines available to your, except
full-screen I/O.

Use PipeThink.  You'll never go back.

OREXXMan
JCL is the buggy whip of 21st century computing.  Stabilize it.
Put Pipelines in the z/OS base.  Would you rather process data in move mode
or locate mode?
IBM has been looking for an HLL for program products; REXX is that language.


On Wed, Sep 23, 2020 at 1:23 PM Martin Packer 
wrote:

> No, that would have to be another job, also specifying eg SUBSYS=PIPE. And
> scheduled to run alongside the sort.
>
> Cheers, Martin
>
> Martin Packer
>
> Systems Investigator & Performance Troubleshooter, IBM
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: https://mainframeperformancetopics.com
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
> https://anchor.fm/marna-walle
>
> Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
>
>
> From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   23/09/2020 17:18
> Subject:[EXTERNAL] Re: SORT question: How to both remove records
> from a file AND write removed records to a 2nd output?
> Sent by:IBM Mainframe Discussion List 
>
>
>
> On Wed, 23 Sep 2020 16:01:08 +0100, Martin Packer wrote:
>
> >Right. That example used BatchPipes/MVS pipes - but it's not essential to
> >the use of OUTFIL SPLIT. The reason it used pipes was to enable cloning
> to
> >balance a pipe.
> >
> OK.  I looked at your example (z/OS 2.1; not too misty).  But what
> mystifies
> me is where to code the Pipeline specification.  Suppose I want to, for
> example to direct OUT1 (or an added OUT3) to a stage such as
> DIGEST MD5.  Could I just code:
>...,SUBSYS='PIPE,DIGEST MD5',...
> ??? if not, how?  And can I further direct that digest to SYSOUT?
>
> >-Original Message-
> >From: IBM Martin Packer
> >Sent: Wednesday, September 23, 2020 4:31 AM
> >...
> >Something also from the mists of time, by no coincidence, is this:
> >
>
> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm
>
>
> -- gil
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Batchpipes (was: SORT question: ...)

[Hobart Spitz]

They find each other via the name of the pipe.


OREXXMan
JCL is the buggy whip of 21st century computing.  Stabilize it.
Put Pipelines in the z/OS base.  Would you rather process data in move mode
or locate mode?
IBM has been looking for an HLL for program products; REXX is that language.


On Wed, Sep 23, 2020 at 2:19 PM Martin Packer 
wrote:

> The pipe name is a data set name.
>
> The scheduling problem is real: How to ensure two jobs run at the same
> time. I would propose common predecessors and common successors. Some of
> these might be dummy operations.
>
> Actually, sorts are nice cases as they often read data that has been
> freshly created or create data that is just about to be read sequentially.
>
> Cheers, Martin
>
> Martin Packer
>
> Systems Investigator & Performance Troubleshooter, IBM
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: https://mainframeperformancetopics.com
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
> https://anchor.fm/marna-walle
>
> Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
>
>
> From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
> To:     IBM-MAIN@LISTSERV.UA.EDU
> Date:   23/09/2020 20:04
> Subject:[EXTERNAL] Batchpipes (was: SORT question: ...)
> Sent by:IBM Mainframe Discussion List 
>
>
>
> On Wed, 23 Sep 2020 19:23:33 +0100, Martin Packer wrote:
>
> >No, that would have to be another job, also specifying eg SUBSYS=PIPE.
> And
> >scheduled to run alongside the sort.
> >
> Eek!  How do they find each other?  And perhaps more important, what
> if several programmers unknown to each other concurrently submit
> job using SUBSYS=PIPE; how do they avoid mistaken identity?
> Must a scheduler sort it out?
>
> I've had corresponding problems with sockets and dynamically assigned
> ports: someone needs to note the port number and communicate it to
> the other.
>
> >>-Original Message-
> >>From: IBM Martin Packer
> >>Sent: Wednesday, September 23, 2020 4:31 AM
> >>...
> >>Something also from the mists of time, by no coincidence, is this:
> >>
> >
>
> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm
>
>
> -- gil
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Batchpipes (was: SORT question: ...)

[Martin Packer]

The pipe name is a data set name.

The scheduling problem is real: How to ensure two jobs run at the same 
time. I would propose common predecessors and common successors. Some of 
these might be dummy operations.

Actually, sorts are nice cases as they often read data that has been 
freshly created or create data that is just about to be read sequentially.

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   23/09/2020 20:04
Subject:[EXTERNAL] Batchpipes (was: SORT question: ...)
Sent by:IBM Mainframe Discussion List 



On Wed, 23 Sep 2020 19:23:33 +0100, Martin Packer wrote:

>No, that would have to be another job, also specifying eg SUBSYS=PIPE. 
And
>scheduled to run alongside the sort.
>
Eek!  How do they find each other?  And perhaps more important, what
if several programmers unknown to each other concurrently submit
job using SUBSYS=PIPE; how do they avoid mistaken identity?
Must a scheduler sort it out?

I've had corresponding problems with sockets and dynamically assigned
ports: someone needs to note the port number and communicate it to
the other.

>>-Original Message-
>>From: IBM Martin Packer
>>Sent: Wednesday, September 23, 2020 4:31 AM
>>...
>>Something also from the mists of time, by no coincidence, is this:
>>
>
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm


-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Batchpipes (was: SORT question: ...)

[Paul Gilmartin]

On Wed, 23 Sep 2020 19:23:33 +0100, Martin Packer wrote:

>No, that would have to be another job, also specifying eg SUBSYS=PIPE. And
>scheduled to run alongside the sort.
>
Eek!  How do they find each other?  And perhaps more important, what
if several programmers unknown to each other concurrently submit
job using SUBSYS=PIPE; how do they avoid mistaken identity?
Must a scheduler sort it out?

I've had corresponding problems with sockets and dynamically assigned
ports: someone needs to note the port number and communicate it to
the other.

>>-Original Message-
>>From: IBM Martin Packer
>>Sent: Wednesday, September 23, 2020 4:31 AM
>>...
>>Something also from the mists of time, by no coincidence, is this:
>>
>https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Martin Packer]

No, that would have to be another job, also specifying eg SUBSYS=PIPE. And 
scheduled to run alongside the sort.

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   23/09/2020 17:18
Subject:[EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



On Wed, 23 Sep 2020 16:01:08 +0100, Martin Packer wrote:

>Right. That example used BatchPipes/MVS pipes - but it's not essential to
>the use of OUTFIL SPLIT. The reason it used pipes was to enable cloning 
to
>balance a pipe.
> 
OK.  I looked at your example (z/OS 2.1; not too misty).  But what 
mystifies
me is where to code the Pipeline specification.  Suppose I want to, for
example to direct OUT1 (or an added OUT3) to a stage such as
DIGEST MD5.  Could I just code:
   ...,SUBSYS='PIPE,DIGEST MD5',...
??? if not, how?  And can I further direct that digest to SYSOUT?

>-Original Message-
>From: IBM Martin Packer
>Sent: Wednesday, September 23, 2020 4:31 AM
>...
>Something also from the mists of time, by no coincidence, is this:
>
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm


-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Paul Gilmartin]

On Wed, 23 Sep 2020 16:01:08 +0100, Martin Packer wrote:

>Right. That example used BatchPipes/MVS pipes - but it's not essential to
>the use of OUTFIL SPLIT. The reason it used pipes was to enable cloning to
>balance a pipe.
> 
OK.  I looked at your example (z/OS 2.1; not too misty).  But what mystifies
me is where to code the Pipeline specification.  Suppose I want to, for
example to direct OUT1 (or an added OUT3) to a stage such as
DIGEST MD5.  Could I just code:
   ...,SUBSYS='PIPE,DIGEST MD5',...
??? if not, how?  And can I further direct that digest to SYSOUT?

>-Original Message-
>From: IBM Martin Packer
>Sent: Wednesday, September 23, 2020 4:31 AM
>...
>Something also from the mists of time, by no coincidence, is this:
>https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Martin Packer]

Right. That example used BatchPipes/MVS pipes - but it's not essential to 
the use of OUTFIL SPLIT. The reason it used pipes was to enable cloning to 
balance a pipe.

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "Farley, Peter x23353" 
<031df298a9da-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   23/09/2020 14:54
Subject:    [EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



That might indeed be interesting if I knew what SUBSYS=PIPE was.  Is that 
the z/OS BatchPipes product?  If so I do not have that available here.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf 
Of Martin Packer
Sent: Wednesday, September 23, 2020 4:31 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT question: How to both remove records from a file AND 
write removed records to a 2nd output?

EXTERNAL EMAIL

OUTFIL SPLIT.

Something also from the mists of time, by no coincidence, is this: 
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm
 


TBH, I probably wrote that and Frank made it worth reading. :-)

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: 
https://mainframeperformancetopics.com 


Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle 


Youtube channel: 
https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA 




From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   22/09/2020 20:31
Subject:    [EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



On Tue, 22 Sep 2020 20:08:12 +0100, Martin Packer wrote:
>
>SAVE has, to my mind, been most invaluable. Prior to this you had to 
>figure out the boolean complement of the INCLUDE= clause. With more 
>than one INCLUDE= clause in a sort it became hairier, requiring De 
>Morgan's Law*. :-)  SAVE made this a lot simpler and less error prone.
> 
Yet adding a NOT operator to the grammar (no one has mentioned that there 
is one) would relieve programmers of the chore of rewriting complex 
Boolean expressions with De Morgan's Law.

>"Card Dealer" not so much.
> 
I don't know "Card Dealer".  Does that amount to having multiple OUTFILs 
using different tags?

Thanks,
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email 
to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the 
addressee and may contain information that is privileged and confidential. 
If the reader of the message is not the intended recipient or an 
authorized representative of the intended recipient, you are hereby 
notified that any dissemination of this communication is strictly 
prohibited. If you have received this communication in error, please 
notify us immediately by e-mail and delete the message and any attachments 
from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Farley, Peter x23353]

That might indeed be interesting if I knew what SUBSYS=PIPE was.  Is that the 
z/OS BatchPipes product?  If so I do not have that available here.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Martin Packer
Sent: Wednesday, September 23, 2020 4:31 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT question: How to both remove records from a file AND write 
removed records to a 2nd output?

EXTERNAL EMAIL

OUTFIL SPLIT.

Something also from the mists of time, by no coincidence, is this: 
https://urldefense.com/v3/__https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!djsBTdtRWhlMDjqIDWZ3N2tii2HhPjYswgwHMqZah3Ae3uJs0JFjTuTVRzoFQonkEIOCWw$
 

TBH, I probably wrote that and Frank made it worth reading. :-)

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: 
https://urldefense.com/v3/__https://mainframeperformancetopics.com__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!djsBTdtRWhlMDjqIDWZ3N2tii2HhPjYswgwHMqZah3Ae3uJs0JFjTuTVRzoFQokXweXXaQ$
 

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://urldefense.com/v3/__https://anchor.fm/marna-walle__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!djsBTdtRWhlMDjqIDWZ3N2tii2HhPjYswgwHMqZah3Ae3uJs0JFjTuTVRzoFQokxmRitYQ$
 

Youtube channel: 
https://urldefense.com/v3/__https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!djsBTdtRWhlMDjqIDWZ3N2tii2HhPjYswgwHMqZah3Ae3uJs0JFjTuTVRzoFQonXW_nH9A$
 



From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   22/09/2020 20:31
Subject:[EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



On Tue, 22 Sep 2020 20:08:12 +0100, Martin Packer wrote:
>
>SAVE has, to my mind, been most invaluable. Prior to this you had to 
>figure out the boolean complement of the INCLUDE= clause. With more 
>than one INCLUDE= clause in a sort it became hairier, requiring De 
>Morgan's Law*. :-)  SAVE made this a lot simpler and less error prone.
> 
Yet adding a NOT operator to the grammar (no one has mentioned that there is 
one) would relieve programmers of the chore of rewriting complex Boolean 
expressions with De Morgan's Law.

>"Card Dealer" not so much.
> 
I don't know "Card Dealer".  Does that amount to having multiple OUTFILs using 
different tags?

Thanks,
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Martin Packer]

Sometimes "in a parallel universe" can mean "in this universe"... :-)

It would be poacher turned, ahem, poacher... :-)

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   Sri h Kolusu 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   22/09/2020 20:34
Subject:    [EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



> On the "tagging" that Sri Hari mentioned:
>
> 1) You needn't tag unless you want to - and there will be some occasions
> where you don't want to.

Matrin,

Fully agreed.  I just wanted to show case with IFTHEN it is much easier to
split into multiple files with different conditions.

> Just some thoughts from someone who's loved playing with what DFSORT can
> do but has never actually been a DFSORT developer.


You are always welcome to join DFSORT.


Thanks,
Kolusu


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Martin Packer]

OUTFIL SPLIT.

Something also from the mists of time, by no coincidence, is this: 
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.icea100/ice2ca_Example_11._Sort_with_Pipes_and_OUTFIL_SPLIT.htm

TBH, I probably wrote that and Frank made it worth reading. :-)

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   22/09/2020 20:31
Subject:[EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



On Tue, 22 Sep 2020 20:08:12 +0100, Martin Packer wrote:
>
>SAVE has, to my mind, been most invaluable. Prior to this you had to
>figure out the boolean complement of the INCLUDE= clause. With more than
>one INCLUDE= clause in a sort it became hairier, requiring De Morgan's
>Law*. :-)  SAVE made this a lot simpler and less error prone.
> 
Yet adding a NOT operator to the grammar (no one has mentioned that
there is one) would relieve programmers of the chore of rewriting complex
Boolean expressions with De Morgan's Law.

>"Card Dealer" not so much.
> 
I don't know "Card Dealer".  Does that amount to having multiple OUTFILs
using different tags?

Thanks,
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Sri h Kolusu]

> On the "tagging" that Sri Hari mentioned:
>
> 1) You needn't tag unless you want to - and there will be some occasions
> where you don't want to.

Matrin,

Fully agreed.  I just wanted to show case with IFTHEN it is much easier to
split into multiple files with different conditions.

> Just some thoughts from someone who's loved playing with what DFSORT can
> do but has never actually been a DFSORT developer.


You are always welcome to join DFSORT.


Thanks,
Kolusu


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Paul Gilmartin]

On Tue, 22 Sep 2020 20:08:12 +0100, Martin Packer wrote:
>
>SAVE has, to my mind, been most invaluable. Prior to this you had to
>figure out the boolean complement of the INCLUDE= clause. With more than
>one INCLUDE= clause in a sort it became hairier, requiring De Morgan's
>Law*. :-)  SAVE made this a lot simpler and less error prone.
> 
Yet adding a NOT operator to the grammar (no one has mentioned that
there is one) would relieve programmers of the chore of rewriting complex
Boolean expressions with De Morgan's Law.

>"Card Dealer" not so much.
> 
I don't know "Card Dealer".  Does that amount to having multiple OUTFILs
using different tags?

Thanks,
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Martin Packer]

Back in the mists of time - so I can say it now :-) - Frank Yaeger and I 
were discussing how we might add some things to the first offering of 
OUTFIL in DFSORT that weren't then in Syncsort. We came up with SAVE and 
"Card Dealer".

SAVE has, to my mind, been most invaluable. Prior to this you had to 
figure out the boolean complement of the INCLUDE= clause. With more than 
one INCLUDE= clause in a sort it became hairier, requiring De Morgan's 
Law*. :-)  SAVE made this a lot simpler and less error prone.

"Card Dealer" not so much.

I would expect both implementations - DFSORT and Syncsort - to have had 
these functions since the 1990s and both have gone much further.

On the "tagging" that Sri Hari mentioned:

1) You needn't tag unless you want to - and there will be some occasions 
where you don't want to. One example might be if you've been counting with 
SUM FIELDS= and a binary "ONE" field. You might throw away records where 
the count is less than, say, 100.

2) One other thing you might want to do is to throw away, say, 90% of the 
records on the way in with INCLUDE/OMIT and then divide up the surviving 
records on the way out, using OUTFIL INCLUDE=/OMIT= and SAVE.

Just some thoughts from someone who's loved playing with what DFSORT can 
do but has never actually been a DFSORT developer.

* Augustus De Morgan founded the Maths Department at UCL, where I learnt a 
trick or two. :-)

Cheers, Martin

Martin Packer

Systems Investigator & Performance Troubleshooter, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   Sri h Kolusu 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   22/09/2020 19:24
Subject:[EXTERNAL] Re: SORT question: How to both remove records 
from a file AND write removed records to a 2nd output?
Sent by:IBM Mainframe Discussion List 



> In one pass of an input file is there a way to use SORT to both
> remove identified records from an input file AND write those removed
> records to a second output file?


Peter,

You don't need 2 passes of data, you can tag the records that match your
identification and then use that tag to filter the records on OUTFIL. Some
thing like this

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC  - DROP THIS RECORD
DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD
//PICKREC  DD SYSOUT=*
//DROPREC  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=(1,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(81:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(81,1,CH,EQ,C'A'),
  BUILD=(1,80)

  OUTFIL FNAMES=DROPREC,SAVE,
  BUILD=(1,80)
/*


The PICKREC file will have the following records

DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD

The DROPREC file will have the following records

ABC  - DROP THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD


For VB files, make sure that you have the Indicator right after the RDW so
that you retain the variable length records as is

Something like this

//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,
 BUILD=(1,4,$ RDW
X,  $ ID TAG
5)),$ DATA AS-IS

IFTHEN=(WHEN=(6,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(05:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(05,1,CH,EQ,C'A'),
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS

  OUTFIL FNAMES=DROPREC,SAVE,
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS
/*


Fruther if you have any questions please let me know

Thanks,
 Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Farley, Peter x23353]

Thanks Sri, that is just what I need.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: Tuesday, September 22, 2020 2:24 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT question: How to both remove records from a file AND write 
removed records to a 2nd output?

> In one pass of an input file is there a way to use SORT to both remove 
> identified records from an input file AND write those removed records 
> to a second output file?


Peter,

You don't need 2 passes of data, you can tag the records that match your 
identification and then use that tag to filter the records on OUTFIL. Some 
thing like this

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC  - DROP THIS RECORD
DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD
//PICKREC  DD SYSOUT=*
//DROPREC  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=(1,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(81:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(81,1,CH,EQ,C'A'),
  BUILD=(1,80)

  OUTFIL FNAMES=DROPREC,SAVE,
  BUILD=(1,80)
/*


The PICKREC file will have the following records

DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD

The DROPREC file will have the following records

ABC  - DROP THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD


For VB files, make sure that you have the Indicator right after the RDW so that 
you retain the variable length records as is

Something like this

//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,
 BUILD=(1,4,$ RDW
X,  $ ID TAG
5)),$ DATA AS-IS

IFTHEN=(WHEN=(6,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(05:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(05,1,CH,EQ,C'A'),
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS

  OUTFIL FNAMES=DROPREC,SAVE,
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS
/*

--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Sri h Kolusu]

> In one pass of an input file is there a way to use SORT to both
> remove identified records from an input file AND write those removed
> records to a second output file?


Peter,

You don't need 2 passes of data, you can tag the records that match your
identification and then use that tag to filter the records on OUTFIL. Some
thing like this

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC  - DROP THIS RECORD
DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD
//PICKREC  DD SYSOUT=*
//DROPREC  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=(1,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(81:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(81,1,CH,EQ,C'A'),
  BUILD=(1,80)

  OUTFIL FNAMES=DROPREC,SAVE,
  BUILD=(1,80)
/*


The PICKREC file will have the following records

DEF  - PICK THIS RECORD
EFG  - PICK THIS RECORD

The DROPREC file will have the following records

ABC  - DROP THIS RECORD
XYZ  - DROP THIS RECORD
ZZZ  - DROP THIS RECORD


For VB files, make sure that you have the Indicator right after the RDW so
that you retain the variable length records as is

Something like this

//SYSINDD *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,
 BUILD=(1,4,$ RDW
X,  $ ID TAG
5)),$ DATA AS-IS

IFTHEN=(WHEN=(6,3,SS,EQ,C'DEF,EFG'),
   OVERLAY=(05:C'A'))

  OUTFIL FNAMES=PICKREC,
  INCLUDE=(05,1,CH,EQ,C'A'),
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS

  OUTFIL FNAMES=DROPREC,SAVE,
BUILD=(1,04,$ RDW
   6)   $ DATA AS-IS
/*


Fruther if you have any questions please let me know

Thanks,
 Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Farley, Peter x23353]

Thanks Max, that would seem to be the way.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Massimo Biancucci
Sent: Tuesday, September 22, 2020 1:44 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT question: How to both remove records from a file AND write 
removed records to a 2nd output?

Peter,

look at OUTFIL syntax and SAVE keyword.

Regards.
Max

> In one pass of an input file is there a way to use SORT to both remove 
> identified records from an input file AND write those removed records 
> to a second output file?
>
> I know I can run two passes of the input using INCLUDE in pass 1 to 
> select just the records to be removed and a second pass using OMIT to 
> take them out, but can it be done in one pass?
>
> Peter
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Massimo Biancucci]

Peter,

look at OUTFIL syntax and SAVE keyword.

Regards.
Max


Mail
priva di virus. www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Il giorno mar 22 set 2020 alle ore 19:22 Farley, Peter x23353 <
031df298a9da-dmarc-requ...@listserv.ua.edu> ha scritto:

> In one pass of an input file is there a way to use SORT to both remove
> identified records from an input file AND write those removed records to a
> second output file?
>
> I know I can run two passes of the input using INCLUDE in pass 1 to select
> just the records to be removed and a second pass using OMIT to take them
> out, but can it be done in one pass?
>
> Peter
>
> This message and any attachments are intended only for the use of the
> addressee and may contain information that is privileged and confidential.
> If the reader of the message is not the intended recipient or an authorized
> representative of the intended recipient, you are hereby notified that any
> dissemination of this communication is strictly prohibited. If you have
> received this communication in error, please notify us immediately by
> e-mail and delete the message and any attachments from your system.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SORT question: How to both remove records from a file AND write removed records to a 2nd output?

[Farley, Peter x23353]

In one pass of an input file is there a way to use SORT to both remove 
identified records from an input file AND write those removed records to a 
second output file?

I know I can run two passes of the input using INCLUDE in pass 1 to select just 
the records to be removed and a second pass using OMIT to take them out, but 
can it be done in one pass?

Peter

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

BLOCK CONTAINS on non-VSAM was Re: Sort Question

[Clark Morris]

[Default] On 11 Oct 2017 05:33:46 -0700, in bit.listserv.ibm-main
peter.far...@broadridge.com (Farley, Peter x23353) wrote:

>It depends on the version of COBOL in use.  COBOL V4.2 and earlier default to 
>BLOCK CONTAINS 1 if the phrase is not specified in the FD, which is why most 
>sensible z/OS shop standards call for always using BLOCK CONTAINS 0.  I can't 
>speak for VM or VSE shop standards, I haven't worked in either of those 
>environments for quite a long time.
>
>COBOL V5+ have a new compiler option, BLOCK0, which changes the default to 
>BLOCK CONTAINS 0, though the option is off by default for compatibility with 
>earlier releases.

If a block count other than zero for a QSAM file is either defaulted
to or explicitly stated (BLOCK 10 for example), then COBOL will
require that no more than that number of blocks is on the input file
and abend if it isn't (I'm not certain how it would handle consecutive
short blocks) unless STATUS-CODE is specified for the file.  On output
the blocksize would be determined by the number of blocks specified if
not zero.  

VSAM files never have had block count specified and have always used
the CISIZE defined.

Clark Morris
>
>HTH
>
>Peter
>
>-Original Message-
>From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
>Behalf Of Tim Hare
>Sent: Tuesday, October 10, 2017 11:19 PM
>To: IBM-MAIN@LISTSERV.UA.EDU
>Subject: Re: Sort Question
>
>I know I am late to this, but I see no BLOCK CONTAINS 0 RECORDS in the COBOL.  
>I'm not totally current on COBOL releases, but 
>A) is that still required to use block size from the JCL
>B) is it even relevant in this instance - COBOL will use the block size value 
>of the dataset that's input, yes?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Farley, Peter x23353]

It depends on the version of COBOL in use.  COBOL V4.2 and earlier default to 
BLOCK CONTAINS 1 if the phrase is not specified in the FD, which is why most 
sensible z/OS shop standards call for always using BLOCK CONTAINS 0.  I can't 
speak for VM or VSE shop standards, I haven't worked in either of those 
environments for quite a long time.

COBOL V5+ have a new compiler option, BLOCK0, which changes the default to 
BLOCK CONTAINS 0, though the option is off by default for compatibility with 
earlier releases.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tim Hare
Sent: Tuesday, October 10, 2017 11:19 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

I know I am late to this, but I see no BLOCK CONTAINS 0 RECORDS in the COBOL.  
I'm not totally current on COBOL releases, but 
A) is that still required to use block size from the JCL
B) is it even relevant in this instance - COBOL will use the block size value 
of the dataset that's input, yes?

--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Tue, 10 Oct 2017 22:18:47 -0500, Tim Hare wrote:

>I know I am late to this, but I see no BLOCK CONTAINS 0 RECORDS in the COBOL.  
>I'm not totally current on COBOL releases, but 
>A) is that still required to use block size from the JCL
>
Even better, it will use the block size supplied by SDB.

>B) is it even relevant in this instance - COBOL will use the block size value 
>of the dataset that's input, yes?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Tim Hare]

I know I am late to this, but I see no BLOCK CONTAINS 0 RECORDS in the COBOL.  
I'm not totally current on COBOL releases, but 
A) is that still required to use block size from the JCL
B) is it even relevant in this instance - COBOL will use the block size value 
of the dataset that's input, yes?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[R D Boenig]

Thanx Dave! 




From:   David Betten <bet...@us.ibm.com>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   10/02/2017 09:45 AM
Subject:    Re: Sort Question
Sent by:IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>



I see a couple things the DFSORT output

The reason for the small mainsize and DSA=0 is because the program is
passing MAINSIZE=00128000

ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS
FOLLOW

  SORT FIELDS=(0001,0013,CH,A)

  RECORD TYPE=F,LENGTH=(49,,)

  OPTION MAINSIZE=00128000



The small mainsize is leading to an intermediate merge.

ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED


An intermediate merge happens when the sortworks are so fragmented, DFSORT
must do an intermediate merge before it can do the final merge to SORTOUT.
This greatly increases the sort work requirement as well as the elapsed
time.   So the earlier recommendation of passing MAINSIZE=MAX via SORTCNTL
or DFSPARM should resolve this.


Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email:  bet...@us.ibm.com


IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
10/02/2017 03:57:35 PM:

> From: "Pfister, Nathan" <nathanpfis...@donegalgroup.com>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 10/02/2017 03:57 PM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>

>
> Sorry for the long post, but here's all the info.
>
> Here's the sort parameters:
>
> ITEM   JCL (ICEAM1) VALUE   INV (ICEAM2) VALUE   TSO
> (ICEAM3) VALUE   TSOINV (ICEAM4) VALUE
> --  
>  -
> ENABLE TD1  NONE TD1
NONE
>* NONE* NONE
>
> ABCODE MSG  MSG  MSG
MSG
> ALTSEQ SEE BELOWSEE BELOWSEE
> BELOWSEE BELOW
> ARESALL000
0
> ARESINVNOT APPLICABLE   0NOT
> APPLICABLE   0
> CFWYES  YES  YES
YES
> CHALT  NO   NO   NO
NO
> CHECK  YES  YES  YES
YES
> CINV   YES  YES  YES
YES
> COBEXITCOB2 COB2
> COB2 COB2
> DIAGSIMNO   NO   NO
NO
> DSA128  128  128
128
> DSPSIZEMAX  MAX  MAX
MAX
> DYNALOC(SYSDA,12)   (SYSDA,12)
> (SYSDA,8)(SYSDA,8)
>* (SYSDA,4)  * (SYSDA,4)  *
> (SYSDA,4)  * (SYSDA,4)
> DYNAPCT75   75   10
10
>* 10 * 10
> DYNAUTOYES  YES  YES
YES
> DYNSPC 1024 1024 256
256
>* 256* 256
> EFSNONE NONE
> NONE NONE
> EQUALS YES  YES  YES
YES
>* VLBLKSET   * VLBLKSET   *
> VLBLKSET   * VLBLKSET
> ERET   ABENDABEND
> RC16 RC16
>* RC16   * RC16
> ESTAE  YES  YES  YES
YES
> EXITCK STRONG   STRONG
> STRONG   STRONG
> EXPMAX MAX  MAX  MAX
MAX
> EXPOLD 50%  50%  50%
50%
> EXPRES 10%  10%  10%
10%
> FSZEST NO   NO   NO
NO
> GENER  NOT APPLICABLE   IEBGENR  NOT
> APPLICABLE   IEBGENR
> GNPAD  NOT APPLICABLE   RC0  NOT
> APPLICABLE   RC0
> GNTRUNCNOT APPLICABLE   RC0  NOT
> APPLICABLE   RC0
> HIPRMAXOPTIMAL  OPTIMAL
> OPTIMAL  OPTIMAL
> IDRCPCTNONE NONE
> NONE NONE
> IEXIT  NO

Re: Sort Question

[Vernooij, Kees (ITOPT1) - KLM]

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Paul Gilmartin
> Sent: 03 October, 2017 15:28
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
> 
> On Tue, 3 Oct 2017 06:42:38 +, Vernooij, Kees (ITOPT1) - KLM wrote:
> >> >
> >> But does this contend adversely with concurrent jobs?
> >
> >DFSORT communicates with RSM in order to utilize memory efficiently,
> without overloading it.
> >
> Then might installations' second-guessing result in imposing static
> resource
> limits that thwart DFSORT's optimization and actually degrade
> performance?
> 
> -- gil
> 

AFAIK, one can only make DFSORT run worse, not better than it would do by 
itself. E.g. we had several problems with users that were experimenting with 
number and sizes of SORTWKs and other parameters in order to get jobs running 
reliably and these problems were usually solved by allowing DFSORT to make its 
own calculations of memory, memobjects, hiper/dataspaces and sortwks.

I hope David Betten cares to elaborate on this.

Kees

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Tue, 3 Oct 2017 06:42:38 +, Vernooij, Kees (ITOPT1) - KLM wrote:
>> >
>> But does this contend adversely with concurrent jobs?
>
>DFSORT communicates with RSM in order to utilize memory efficiently, without 
>overloading it.
> 
Then might installations' second-guessing result in imposing static resource
limits that thwart DFSORT's optimization and actually degrade performance?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Vernooij, Kees (ITOPT1) - KLM]

> >
> But does this contend adversely with concurrent jobs?
> 


DFSORT communicates with RSM in order to utilize memory efficiently, without 
overloading it.

Kees.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Pfister, Nathan]

Yes, the MOSIZE went from MAX to 0.  This was based on an APAR from V1R10 which 
the app prog saw and decided to try.  I honestly was shocked it worked.

Here's the APAR for reference:
http://www-01.ibm.com/support/docview.wss?uid=isg1PM23527#more



-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Elardus Engelbrecht
Sent: Monday, October 02, 2017 3:33 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

Pfister, Nathan wrote:

>That said, by setting MOSIZE to 0...it fixed the issue as was with no other 
>changes.  Right, wrong, or indifferent.

Interesting. You changed to 0 from MOSIZE = MAX (from you earlier post where 
you listed the sort parameters)?

Hmmm, interesting, but thanks. Good to know about it.

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance Group may 
contain CONFIDENTIAL and legally protected information. If you are not an 
intended recipient, please do not copy, use or disclose this email or its 
contents to others; and please notify us by calling toll free (800) 877-0600 
x7880 or by replying to this message, and then delete it from your system. 
Delivery of this email to an unintended recipient is not a waiver of any 
attorney-client or other applicable privilege.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[David W Noon]

On Mon, 2 Oct 2017 13:02:24 -0500, Elardus Engelbrecht
(elardus.engelbre...@sita.co.za) wrote about "Re: Sort Question" (in
<4916708819685726.wa.elardus.engelbrechtsita.co...@listserv.ua.edu>):

[snip]
> AFAIK, COBOL has its own internal SORT functions,

It does not. Both PL/I and COBOL use the SORT or ICEMAN module from the
linklist for their sort interfaces.

> but if I want to sort something big or complex, I would terminate my
> COBOL program, let sort it with DFSORT or SyncSort and do same or
> other COBOL program to process the sorted data.
> 
> If it is not too big or complex to sort, I can let COBOL call DFSORT
> and wait for sorted data to come back. 

Unless your COBOL code is huge, it makes little difference to the
performance of the sort. Just use the SORT verb in COBOL or the
PLISRTx() subroutines in PL/I; they will perform just the same as a
separate sort step. You need to remember to set the appropriate memory
parameters in PLSIRTx() or set the COBOL special registers so that the
sort can use as much memory as it needs. These compiler language
interfaces have all the facilities of DF/SORT available, including the
SORTPARM DD card stream.

>> Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect
> that in the era of oscillating merge it relied on EXCP.
> 
> AFAIK, DFSORT has its own OCO access method(s) just like RACF has
> its own OCO way to access the RACF database.

DF/SORT has used EXCP since it was called SORT/360.
Note that if SORTIN or SORTOUT is a VSAM cluster, it will use normal
VSAM for reading or writing, but SORTWKxx spill files are accessed by
raw channel programs (at least when the default BLOCKSET algorithm is used).
-- 
Regards,

Dave  [RLU #314465]
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
david.w.n...@googlemail.com (David W Noon)
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*

 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Elardus Engelbrecht]

Pfister, Nathan wrote:

>That said, by setting MOSIZE to 0...it fixed the issue as was with no other 
>changes.  Right, wrong, or indifferent.

Interesting. You changed to 0 from MOSIZE = MAX (from you earlier post where 
you listed the sort parameters)?

Hmmm, interesting, but thanks. Good to know about it.

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Pfister, Nathan]

I'm working with the app team to get some changes in place for memory 
utilization based on some things given me here.

That said, by setting MOSIZE to 0...it fixed the issue as was with no other 
changes.  Right, wrong, or indifferent.

I do thank everyone for their input on this!

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Clark Morris
Sent: Monday, October 02, 2017 3:19 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

[Default] On 2 Oct 2017 11:01:13 -0700, in bit.listserv.ibm-main 
elardus.engelbre...@sita.co.za (Elardus Engelbrecht) wrote:

>Paul Gilmartin wrote:
>
>You have made good points, thanks.
>
>
>>> With respect to the REGION size, remember that SORT (both IBM and its 
>>> competitors) is most efficient the more memory you can let him have.  I 
>>> have found this especially true for COBOL-based SORT's.
>
>>But does this contend adversely with concurrent jobs?
>
>Uhm, with same or different sort input and/or output for all these concurrent 
>jobs?
>
>
>AFAIK, COBOL has its own internal SORT functions, but if I want to sort 
>something big or complex, I would terminate my COBOL program, let sort it with 
>DFSORT or SyncSort and do same or other COBOL program to process the sorted 
>data.

IBM COBOL programs invoke the installation SORT program - DFSORT or SYNCSORT.  
The MAINSIZE of 000128000 seems small if that is in bytes.
I would look at the COBOL compile for that program, the SORT defaults for 
invoked (as opposed to JCL) sorts, any override JCL and all of the installation 
COBOL compiler options to see what is generating that size.

Clark Morris
>
>If it is not too big or complex to sort, I can let COBOL call DFSORT and wait 
>for sorted data to come back.
>
>
>>I would hope (wish) that DFSORT would supply optimum defaults.
>
>AFAIK, those defaults are the best DFSORT can offer, but of course, only you 
>know if those defaults are working or not.
>
>
>>Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
>>era of oscillating merge it relied on EXCP.
>
>AFAIK, DFSORT has its own OCO access method(s) just like RACF has its own OCO 
>way to access the RACF database.
>
>Just my 3 cents, not 2, because of tax/inflation and a terrible
>mother-in-law... ;-)
>
>Groete / Greetings
>Elardus Engelbrecht
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions, send
>email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance Group may 
contain CONFIDENTIAL and legally protected information. If you are not an 
intended recipient, please do not copy, use or disclose this email or its 
contents to others; and please notify us by calling toll free (800) 877-0600 
x7880 or by replying to this message, and then delete it from your system. 
Delivery of this email to an unintended recipient is not a waiver of any 
attorney-client or other applicable privilege.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Clark Morris]

[Default] On 2 Oct 2017 11:01:13 -0700, in bit.listserv.ibm-main
elardus.engelbre...@sita.co.za (Elardus Engelbrecht) wrote:

>Paul Gilmartin wrote:
>
>You have made good points, thanks.
>
>
>>> With respect to the REGION size, remember that SORT (both IBM and its 
>>> competitors) is most efficient the more memory you can let him have.  I 
>>> have found this especially true for COBOL-based SORT's.
>
>>But does this contend adversely with concurrent jobs?
>
>Uhm, with same or different sort input and/or output for all these concurrent 
>jobs?
>
>
>AFAIK, COBOL has its own internal SORT functions, but if I want to sort 
>something big or complex, I would terminate my COBOL program, let sort it with 
>DFSORT or SyncSort and do same or other COBOL program to process the sorted 
>data.

IBM COBOL programs invoke the installation SORT program - DFSORT or
SYNCSORT.  The MAINSIZE of 000128000 seems small if that is in bytes.
I would look at the COBOL compile for that program, the SORT defaults
for invoked (as opposed to JCL) sorts, any override JCL and all of the
installation COBOL compiler options to see what is generating that
size.

Clark Morris
>
>If it is not too big or complex to sort, I can let COBOL call DFSORT and wait 
>for sorted data to come back. 
>
>
>>I would hope (wish) that DFSORT would supply optimum defaults. 
>
>AFAIK, those defaults are the best DFSORT can offer, but of course, only you 
>know if those defaults are working or not.
>
>
>>Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
>>era of oscillating merge it relied on EXCP.
>
>AFAIK, DFSORT has its own OCO access method(s) just like RACF has its own OCO 
>way to access the RACF database.
>
>Just my 3 cents, not 2, because of tax/inflation and a terrible 
>mother-in-law... ;-)
>
>Groete / Greetings
>Elardus Engelbrecht
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Elardus Engelbrecht]

Paul Gilmartin wrote:

You have made good points, thanks.


>> With respect to the REGION size, remember that SORT (both IBM and its 
>> competitors) is most efficient the more memory you can let him have.  I have 
>> found this especially true for COBOL-based SORT's.

>But does this contend adversely with concurrent jobs?

Uhm, with same or different sort input and/or output for all these concurrent 
jobs?


AFAIK, COBOL has its own internal SORT functions, but if I want to sort 
something big or complex, I would terminate my COBOL program, let sort it with 
DFSORT or SyncSort and do same or other COBOL program to process the sorted 
data.

If it is not too big or complex to sort, I can let COBOL call DFSORT and wait 
for sorted data to come back. 


>I would hope (wish) that DFSORT would supply optimum defaults. 

AFAIK, those defaults are the best DFSORT can offer, but of course, only you 
know if those defaults are working or not.


>Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the era 
>of oscillating merge it relied on EXCP.

AFAIK, DFSORT has its own OCO access method(s) just like RACF has its own OCO 
way to access the RACF database.

Just my 3 cents, not 2, because of tax/inflation and a terrible 
mother-in-law... ;-)

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question [AD - Kind of]

[Elardus Engelbrecht]

Blaicher, Christopher Y. wrote:

>Syncsort's MFX sort when using the Global Sort Monitor measures systems 
>resources, both on an instantaneous basis and on a historical basis to judge 
>how much memory each sort should use. 

I wish with a really sore heart that all applications and compiled programs do 
those two tricks automagically and in a jiffy.

... like that windoze operating system which remembers your few last used 
application programs and store it in the memory until the page file(s) gets 
full... You run that application again, those associated programs are already 
in the memory, no need to read in from your hard disk.

On the PCs, these languages, Clarion and Clipper, have background things which 
dynamically optimize your memory and files in use while the MSDOS machines are 
idle... Nice features which proved useful.

I believe on C64 from Commodore does the same thing, it does that infamous 
garbage collection when your toy is idle...

So, something which is used to adjust your own performance while running, yes, 
there is some overhead with those dynamic measurements, but if overall CPU 
usage and I/O usage are fewer, that should be good. 

I know about Linklist, COFVLFxx, XCF structures for RACF and such animals, but 
they are somewhat static ('historical basis'), not dynamic like that SyncSort 
Feature.


>On the other hand if there is paging already happening, you only want the sort 
>to take as much as is absolutely necessary to do the job with relative 
>efficiency.

That is another feature I want to steal Sorry, but is it Friday today? ;-)

Thanks for your notes. Much appreciated.

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question [AD - Kind of]

[Blaicher, Christopher Y.]

Syncsort's MFX sort when using the Global Sort Monitor measures systems 
resources, both on an instantaneous basis and on a historical basis to judge 
how much memory each sort should use.  If you give a MFX sort a REGION=0M it 
may use 8M or 1.5G depending on what the current memory demands are by other 
processes, and what the historical data indicates will be needed over the life 
of the sort.  MFX automatically does all that for you.

If you have a 10G machine and over 2G of it are free, who cares that a sort 
comes along and uses a gig.  It isn't hurting anything.  On the other hand if 
there is paging already happening, you only want the sort to take as much as is 
absolutely necessary to do the job with relative efficiency.

Chris Blaicher
Technical Architect
Mainframe Development
P: 201-930-8234  |  M: 512-627-3803
E: cblaic...@syncsort.com

Syncsort Incorporated
2 Blue Hill Plaza #1563
Pearl River, NY 10965
www.syncsort.com

Data quality leader Trillium Software is now a part of Syncsort.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Farley, Peter x23353
Sent: Monday, October 2, 2017 12:49 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Paul Gilmartin
> Sent: Monday, October 02, 2017 12:24 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
>
> On 2017-10-02, at 09:29, Farley, Peter x23353 wrote:
>
> > With respect to the REGION size, remember that SORT (both IBM and its 
> > competitors) is most efficient the more memory you can let him have.  I 
> > have found this especially true for COBOL-based SORT's.
> >
> But does this contend adversely with concurrent jobs?

IMHO it is the job of the Capacity and Performance team to tell the application 
programming teams when application jobs are contending for memory and causing 
paging, especially if it causes SLA's to be missed or near-missed.

> > Consider using a region of at least 500M for large-volume sorts, and be 
> > sure to tell SORT he can use (most of) it via the SORT parameters.  The 
> > more memory he has available the fewer SORTWK's and SORTWK I/O's he will 
> > need to use.
> >
> REGION=0K?

Only if allowed by IEFUSI.  Many large shops do not allow it except with 
special dispensation and approval, specifically because of the issue you raised 
above about memory contention.

> > Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) 
> > or AMP='BUFND=,BUFNI=,RMODE31=ALL'  (VSAM) parameters (as 
> > appropriate).  Consider using SUBSYS=BLSR or SMB parameters for high-volume 
> > VSAM inputs as well.
> >
> I would hope (wish) that DFSORT would supply optimum defaults.  But can an 
> application discern in OPEN exit whether the options were supplied in JCL or 
> as OS defaults?
>
> Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
> era of oscillating merge it relied on EXCP.

Of course SORT uses his own EXCP (or OCO media manager interfaces) where 
possible, as it is far more efficient than QSAM.  I was referring to the OP's 
original COBOL source for this SORT question.  My reply concerned 
application-read input files, not directly-read-by-SORT files.  For JCL sorts I 
always let SORT decide how to optimize his files, but COBOL SORT's act as E15 
and/or E35 exits, so COBOL I/O can and often is being used for those I/O's, 
whether directly for the file to be sorted or a multiplicity of inputs used to 
construct the SORT input and output records.

> > Memory is (relatively) inexpensive, so use it to your advantage.  Your 
> > SLA's will appreciate it.

Peter
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN





ATTENTION: -

The information contained in this message (including any files transmitted with 
this message) may contain proprietary, trade secret or other confidential 
and/or legally privileged information. Any pricing information contained in 
this message or in any files transmitted with this message is always 
confid

Re: Sort Question

[Paul Gilmartin]

On Mon, 2 Oct 2017 16:48:39 +, Farley, Peter x23353 wrote:
>>
>> Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
>> era of oscillating merge it relied on EXCP.
>
>Of course SORT uses his own EXCP (or OCO media manager interfaces) where 
>possible, as it is far more efficient than QSAM.  ...
> 
I'd expect this for PDSE; perhaps less so for allocated OMVS files.

Binder arrogantly ignores "DD FILEDATA=...".  I'd hope for better from DFSORT.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Farley, Peter x23353]

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Paul Gilmartin
> Sent: Monday, October 02, 2017 12:24 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
>
> On 2017-10-02, at 09:29, Farley, Peter x23353 wrote:
>
> > With respect to the REGION size, remember that SORT (both IBM and its 
> > competitors) is most efficient the more memory you can let him have.  I 
> > have found this especially true for COBOL-based SORT's.
> >  
> But does this contend adversely with concurrent jobs?

IMHO it is the job of the Capacity and Performance team to tell the application 
programming teams when application jobs are contending for memory and causing 
paging, especially if it causes SLA's to be missed or near-missed.

> > Consider using a region of at least 500M for large-volume sorts, and be 
> > sure to tell SORT he can use (most of) it via the SORT parameters.  The 
> > more memory he has available the fewer SORTWK's and SORTWK I/O's he will 
> > need to use.
> >  
> REGION=0K?

Only if allowed by IEFUSI.  Many large shops do not allow it except with 
special dispensation and approval, specifically because of the issue you raised 
above about memory contention.

> > Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) 
> > or AMP='BUFND=,BUFNI=,RMODE31=ALL'  (VSAM) parameters (as 
> > appropriate).  Consider using SUBSYS=BLSR or SMB parameters for high-volume 
> > VSAM inputs as well.
> >  
> I would hope (wish) that DFSORT would supply optimum defaults.  But can an 
> application discern in OPEN exit whether the options were supplied in JCL or 
> as OS defaults?
>
> Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in the 
> era of oscillating merge it relied on EXCP.

Of course SORT uses his own EXCP (or OCO media manager interfaces) where 
possible, as it is far more efficient than QSAM.  I was referring to the OP's 
original COBOL source for this SORT question.  My reply concerned 
application-read input files, not directly-read-by-SORT files.  For JCL sorts I 
always let SORT decide how to optimize his files, but COBOL SORT's act as E15 
and/or E35 exits, so COBOL I/O can and often is being used for those I/O's, 
whether directly for the file to be sorted or a multiplicity of inputs used to 
construct the SORT input and output records.

> > Memory is (relatively) inexpensive, so use it to your advantage.  Your 
> > SLA's will appreciate it.

Peter
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On 2017-10-02, at 09:29, Farley, Peter x23353 wrote:

> With respect to the REGION size, remember that SORT (both IBM and its 
> competitors) is most efficient the more memory you can let him have.  I have 
> found this especially true for COBOL-based SORT's.
>  
But does this contend adversely with concurrent jobs?

> Consider using a region of at least 500M for large-volume sorts, and be sure 
> to tell SORT he can use (most of) it via the SORT parameters.  The more 
> memory he has available the fewer SORTWK's and SORTWK I/O's he will need to 
> use.
>  
REGION=0K?

> Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) 
> or AMP='BUFND=,BUFNI=,RMODE31=ALL'  (VSAM) parameters (as 
> appropriate).  Consider using SUBSYS=BLSR or SMB parameters for high-volume 
> VSAM inputs as well.
>  
I would hope (wish) that DFSORT would supply optimum defaults.  But can
an application discern in OPEN exit whether the options were supplied in
JCL or as OS defaults?

Does DFSORT rely on QSAM or on idiosyncratic EXCP?  I'd expect that in
the era of oscillating merge it relied on EXCP.

> Memory is (relatively) inexpensive, so use it to your advantage.  Your SLA's 
> will appreciate it.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Farley, Peter x23353]

And for your COBOL programmers, if they do not use DFSPARM / $ORTPARM DD 
statements in the JCL to control SORT parameters, there are two COBOL-defined 
"special registers" that can be used to specify storage values.

Below is a copy of the definition of those data areas from the COBOL V5.2 
manual.

HTH

Peter

SORT-CORE-SIZE
The SORT-CORE-SIZE special register is the name of a binary data item that you
can use to specify the number of bytes of storage available to the sort utility.
Restriction: The SORT-CORE-SIZE special register is not applicable to sorting a
table with the format 2 SORT statement.
The SORT-CORE-SIZE special register has the implicit definition:
01 SORT-CORE-SIZE GLOBAL PICTURE S9(8) USAGE BINARY VALUE ZERO.
When used in nested programs, this special register is implicitly defined with 
the
global attribute in the outermost program.
SORT-CORE-SIZE can be used in place of the MAINSIZE or RESINV control
statements in the sort control file:
-- The 'MAINSIZE=' option control statement keyword is equivalent to
SORT-CORE-SIZE with a positive value.
-- The 'RESINV=' option control statement keyword is equivalent to
SORT-CORE-SIZE with a negative value.
-- The 'MAINSIZE=MAX' option control statement keyword is equivalent to
SORT-CORE-SIZE with a value of +99 or +.
You can specify the SORT-CORE-SIZE special register in a function wherever an
integer argument is allowed.

SORT-FILE-SIZE
The SORT-FILE-SIZE special register is the name of a binary data item that you 
can
use to specify the estimated number of records in the sort input file, 
file-name-1.
Restriction: The SORT-FILE-SIZE special register is not applicable to sorting a
table with the format 2 SORT statement.
The SORT-FILE-SIZE special register has the implicit definition:
01 SORT-FILE-SIZE GLOBAL PICTURE S9(8) USAGE BINARY VALUE ZERO.
When used in nested programs, this special register is implicitly defined with 
the
global attribute in the outermost program.
SORT-FILE-SIZE is equivalent to the 'FILSZ=Ennn' control statement in the sort
control file.
You can specify the SORT-FILE-SIZE special register in a function wherever an
integer argument is allowed.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of David Betten
Sent: Monday, October 02, 2017 11:45 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

I see a couple things the DFSORT output

The reason for the small mainsize and DSA=0 is because the program is
passing MAINSIZE=00128000

ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS
FOLLOW

  SORT FIELDS=(0001,0013,CH,A)

  RECORD TYPE=F,LENGTH=(49,,)

  OPTION MAINSIZE=00128000



The small mainsize is leading to an intermediate merge.

ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED


An intermediate merge happens when the sortworks are so fragmented, DFSORT
must do an intermediate merge before it can do the final merge to SORTOUT.
This greatly increases the sort work requirement as well as the elapsed
time.   So the earlier recommendation of passing MAINSIZE=MAX via SORTCNTL
or DFSPARM should resolve this.


Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email:  bet...@us.ibm.com


IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
10/02/2017 03:57:35 PM:

> From: "Pfister, Nathan" <nathanpfis...@donegalgroup.com>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 10/02/2017 03:57 PM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>

>
> Sorry for the long post, but here's all the info.
>
> Here's the sort parameters:
>
> ITEM   JCL (ICEAM1) VALUE   INV (ICEAM2) VALUE   TSO
> (ICEAM3) VALUE   TSOINV (ICEAM4) VALUE
> --  
>  -
> ENABLE TD1  NONE TD1
NONE
>* NONE* NONE
>
> ABCODE MSG  MSG  MSG
MSG
> ALTSEQ SEE BELOWSEE BELOWSEE
> BELOWSEE BELOW
> ARESALL000
0
> ARESINVNOT APPLICABLE   0NOT
> APPLICABLE   0
> CFWYES  YES  YES
YES
> CHALT  NO   NO   NO
NO
> CHECK  YES  YES  YES
YES
> CINV   YES  YES  YES
YES
> COBEXITCOB2 COB2
> COB2 COB2
> DIAGSIMNO  

Re: Sort Question

[David Betten]

I see a couple things the DFSORT output

The reason for the small mainsize and DSA=0 is because the program is
passing MAINSIZE=00128000

ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS
FOLLOW

  SORT FIELDS=(0001,0013,CH,A)

  RECORD TYPE=F,LENGTH=(49,,)

  OPTION MAINSIZE=00128000



The small mainsize is leading to an intermediate merge.

ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED


An intermediate merge happens when the sortworks are so fragmented, DFSORT
must do an intermediate merge before it can do the final merge to SORTOUT.
This greatly increases the sort work requirement as well as the elapsed
time.   So the earlier recommendation of passing MAINSIZE=MAX via SORTCNTL
or DFSPARM should resolve this.


Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email:  bet...@us.ibm.com


IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
10/02/2017 03:57:35 PM:

> From: "Pfister, Nathan" <nathanpfis...@donegalgroup.com>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 10/02/2017 03:57 PM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>

>
> Sorry for the long post, but here's all the info.
>
> Here's the sort parameters:
>
> ITEM   JCL (ICEAM1) VALUE   INV (ICEAM2) VALUE   TSO
> (ICEAM3) VALUE   TSOINV (ICEAM4) VALUE
> --  
>  -
> ENABLE TD1  NONE TD1
NONE
>* NONE* NONE
>
> ABCODE MSG  MSG  MSG
MSG
> ALTSEQ SEE BELOWSEE BELOWSEE
> BELOWSEE BELOW
> ARESALL000
0
> ARESINVNOT APPLICABLE   0NOT
> APPLICABLE   0
> CFWYES  YES  YES
YES
> CHALT  NO   NO   NO
NO
> CHECK  YES  YES  YES
YES
> CINV   YES  YES  YES
YES
> COBEXITCOB2 COB2
> COB2 COB2
> DIAGSIMNO   NO   NO
NO
> DSA128  128  128
128
> DSPSIZEMAX  MAX  MAX
MAX
> DYNALOC(SYSDA,12)   (SYSDA,12)
> (SYSDA,8)(SYSDA,8)
>* (SYSDA,4)  * (SYSDA,4)  *
> (SYSDA,4)  * (SYSDA,4)
> DYNAPCT75   75   10
10
>* 10 * 10
> DYNAUTOYES  YES  YES
YES
> DYNSPC 1024 1024 256
256
>* 256* 256
> EFSNONE NONE
> NONE NONE
> EQUALS YES  YES  YES
YES
>* VLBLKSET   * VLBLKSET   *
> VLBLKSET   * VLBLKSET
> ERET   ABENDABEND
> RC16 RC16
>* RC16   * RC16
> ESTAE  YES  YES  YES
YES
> EXITCK STRONG   STRONG
> STRONG   STRONG
> EXPMAX MAX  MAX  MAX
MAX
> EXPOLD 50%  50%  50%
50%
> EXPRES 10%  10%  10%
10%
> FSZEST NO   NO   NO
NO
> GENER  NOT APPLICABLE   IEBGENR  NOT
> APPLICABLE   IEBGENR
> GNPAD  NOT APPLICABLE   RC0  NOT
> APPLICABLE   RC0
> GNTRUNCNOT APPLICABLE   RC0  NOT
> APPLICABLE   RC0
> HIPRMAXOPTIMAL  OPTIMAL
> OPTIMAL  OPTIMAL
> IDRCPCTNONE NONE
> NONE NONE
> IEXIT  NO   NO   NO
NO
> IGNCKPTYES  YES  YES
YES
> IOMAXBF35651584 35651584
> 35651584 35651584
> LIST   YES  YES  

Re: Sort Question

[Farley, Peter x23353]

With respect to the REGION size, remember that SORT (both IBM and its 
competitors) is most efficient the more memory you can let him have.  I have 
found this especially true for COBOL-based SORT's.

Consider using a region of at least 500M for large-volume sorts, and be sure to 
tell SORT he can use (most of) it via the SORT parameters.  The more memory he 
has available the fewer SORTWK's and SORTWK I/O's he will need to use.

Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) or 
AMP='BUFND=,BUFNI=,RMODE31=ALL'  (VSAM) parameters (as appropriate).  
Consider using SUBSYS=BLSR or SMB parameters for high-volume VSAM inputs as 
well.

Memory is (relatively) inexpensive, so use it to your advantage.  Your SLA's 
will appreciate it.

HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Pfister, Nathan
Sent: Monday, October 02, 2017 11:14 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

Lizette;

Thanks for that suggestion.  For now, I looked at what you have and I will give 
some similar cards a shot.  Also, I didn't realize I could reach out to the 
DFSORT team directly, so thanks for that!

Eldarus;

I hadn't thought about the region, it is limited by the IEFUSI exit.

If the parameters I put in based on Lizettes don’t' work, I will suggest coding 
of the SORTWK statements.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Lizette Koehler
Sent: Monday, October 02, 2017 10:17 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

Some interesting sections from DFSORT messages

   1)  DSA is set to 128 in parms but DSA=0 is in the sort message output
   2)  DFSORT at failure says an additional 44 WORK volumes were needed

There is a way to pass parms to DFSORT using DFSPARM DD statement

For our larger sorts I have coded

  //DFSPARM  DD *
 DSA=128,FILESZ=5000,UNIT=(SYSDA,32)


Maybe the DFSORT team will chime in

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Elardus Engelbrecht
> Sent: Monday, October 02, 2017 7:14 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
>
> Pfister, Nathan wrote:
>
> >Sorry for the long post, but here's all the info.
> >Here's the jobs output:
>
> >ICE092I 0 MAIN STORAGE = (128000,1048576,1048576) ICE156I 0 MAIN
> >STORAGE ABOVE 16MB = (590271,590271)
>
> It is looking small for me, at least for me. What is your job REGION?
> Is that REGION limited by SMFPRMxx or IEFUSI exit?
>
> Consider adding these OPTION SIZE=E9,MAINSIZE=MAX just to tell
> DFSORT there is a large workset waiting for it.
>
> Or you can add a good handfull of these SORTWKxx, which I had to use
> at one stage to handle a Galaxy sized sort work:
>
> //SORTWK01 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(1000,500,0)),
> // UNIT=3390
>
> HTH!
>
> Please let us know if you got a solution!
>
> Groete / Greetings
> Elardus Engelbrecht
>
--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Pfister, Nathan]

Lizette;

Thanks for that suggestion.  For now, I looked at what you have and I will give 
some similar cards a shot.  Also, I didn't realize I could reach out to the 
DFSORT team directly, so thanks for that!

Eldarus;

I hadn't thought about the region, it is limited by the IEFUSI exit.

If the parameters I put in based on Lizettes don’t' work, I will suggest coding 
of the SORTWK statements.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Lizette Koehler
Sent: Monday, October 02, 2017 10:17 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

Some interesting sections from DFSORT messages

   1)  DSA is set to 128 in parms but DSA=0 is in the sort message output
   2)  DFSORT at failure says an additional 44 WORK volumes were needed

There is a way to pass parms to DFSORT using DFSPARM DD statement

For our larger sorts I have coded

  //DFSPARM  DD *
 DSA=128,FILESZ=5000,UNIT=(SYSDA,32)


Maybe the DFSORT team will chime in

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Elardus Engelbrecht
> Sent: Monday, October 02, 2017 7:14 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
>
> Pfister, Nathan wrote:
>
> >Sorry for the long post, but here's all the info.
> >Here's the jobs output:
>
> >ICE092I 0 MAIN STORAGE = (128000,1048576,1048576) ICE156I 0 MAIN
> >STORAGE ABOVE 16MB = (590271,590271)
>
> It is looking small for me, at least for me. What is your job REGION?
> Is that REGION limited by SMFPRMxx or IEFUSI exit?
>
> Consider adding these OPTION SIZE=E9,MAINSIZE=MAX just to tell
> DFSORT there is a large workset waiting for it.
>
> Or you can add a good handfull of these SORTWKxx, which I had to use
> at one stage to handle a Galaxy sized sort work:
>
> //SORTWK01 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(1000,500,0)),
> // UNIT=3390
>
> HTH!
>
> Please let us know if you got a solution!
>
> Groete / Greetings
> Elardus Engelbrecht
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance Group may 
contain CONFIDENTIAL and legally protected information. If you are not an 
intended recipient, please do not copy, use or disclose this email or its 
contents to others; and please notify us by calling toll free (800) 877-0600 
x7880 or by replying to this message, and then delete it from your system. 
Delivery of this email to an unintended recipient is not a waiver of any 
attorney-client or other applicable privilege.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Lizette Koehler]

Some interesting sections from DFSORT messages

   1)  DSA is set to 128 in parms but DSA=0 is in the sort message output
   2)  DFSORT at failure says an additional 44 WORK volumes were needed

There is a way to pass parms to DFSORT using DFSPARM DD statement

For our larger sorts I have coded   

  //DFSPARM  DD *
 DSA=128,FILESZ=5000,UNIT=(SYSDA,32)


Maybe the DFSORT team will chime in

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Elardus Engelbrecht
> Sent: Monday, October 02, 2017 7:14 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
> 
> Pfister, Nathan wrote:
> 
> >Sorry for the long post, but here's all the info.
> >Here's the jobs output:
> 
> >ICE092I 0 MAIN STORAGE = (128000,1048576,1048576) ICE156I 0 MAIN
> >STORAGE ABOVE 16MB = (590271,590271)
> 
> It is looking small for me, at least for me. What is your job REGION? Is that
> REGION limited by SMFPRMxx or IEFUSI exit?
> 
> Consider adding these OPTION SIZE=E9,MAINSIZE=MAX just to tell DFSORT
> there is a large workset waiting for it.
> 
> Or you can add a good handfull of these SORTWKxx, which I had to use at one
> stage to handle a Galaxy sized sort work:
> 
> //SORTWK01 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(1000,500,0)),
> // UNIT=3390
> 
> HTH!
> 
> Please let us know if you got a solution!
> 
> Groete / Greetings
> Elardus Engelbrecht
> 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Elardus Engelbrecht]

Pfister, Nathan wrote:

>Sorry for the long post, but here's all the info.
>Here's the jobs output:

>ICE092I 0 MAIN STORAGE = (128000,1048576,1048576)
>ICE156I 0 MAIN STORAGE ABOVE 16MB = (590271,590271)

It is looking small for me, at least for me. What is your job REGION? Is that 
REGION limited by SMFPRMxx or IEFUSI exit?

Consider adding these OPTION SIZE=E9,MAINSIZE=MAX just to tell DFSORT 
there is a large workset waiting for it.

Or you can add a good handfull of these SORTWKxx, which I had to use at one 
stage to handle a Galaxy sized sort work:

//SORTWK01 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(1000,500,0)),
// UNIT=3390

HTH!

Please let us know if you got a solution!

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Pfister, Nathan]

// COND=(0,LT)
//*
//DATEFLE  DD DSN=(0),
// DISP=SHR
//*
//GTAMLIB  DD DSN=, 0
// DISP=SHR
//*
//RESVIND  DD DSN=(0), 000
// DISP=SHR
//*
//SORTCNTL DD DSN=(),
// DISP=SHR
//*
//DFSPARM  DD DSN=(),
// DISP=SHR
//*
//SYSINDD DUMMY
//*
//PRINTER  DD SYSOUT=(B,)
//*
//SYSOUT   DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SYSABEND DD SYSOUT=*
//*
//*
//*
//*
//*END OF PROC  PRSPM175
//*

The PARM currently is:
   OPTION DYNSPC=1024,DYNALLOC=(SYSDA,59),FILSZ=E6000



-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Vernooij, Kees (ITOPT1) - KLM
Sent: Monday, October 02, 2017 9:47 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sort Question

Apparently your sortwk's are that small, that many more are needed.

Post your sortparameter, displayable with this job:
//LISTDEF EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG  DD SYSOUT=*
//SHOWDEF DD SYSOUT=*
//TOOLIN  DD *
  DEFAULTS LIST(SHOWDEF)
/*

Post the full DFSORT output and the JCL of the step with the error.

This info must give a clue.

Kees.

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Pfister, Nathan
> Sent: 02 October, 2017 15:40
> To: IBM-MAIN@LISTSERV.UA.EDU<mailto:IBM-MAIN@LISTSERV.UA.EDU>
> Subject: Sort Question
>
> List;
>
> I have a question dealing with a DFSORT job which I am hoping to get
> some guidance on.
>
> We have a COBOL program which inconsistently gets a SORT CAPACITY
> EXCEEDED error.  They've changed their cards several times with
> several different options to try and get this to work.  Depending on
> the month sometimes it works and sometimes it doesn't, with their
> typical OPTION
> DYNSPC=1024,DYNALLOC=(SYSDA,59)
>
> The latest run, we added FILSZ=E6000
>
> In every run it seems to exceed the sort capacity.
> +ICE046A 5 PRSPM175.J05  SORT CAPACITY EXCEEDED - RECORD COUNT
> 58081219
>
> And the following error as well.  Not sure
>
> ICE278I 1 59 WORK DATA SETS WERE INSUFFICIENT TO COMPLETE THIS SORT SO
> 44 ADDITIONAL WERE USED
>
> The funny thing is, this has worked with a DYNALLOC of 24...there is
> nothing else in the system at the time this was running, and plenty of
> DASD.  Obviously I am overlooking some key piece of information.
>
> Here's the COBOL for the Sort, which doesn't mean too much to me as I
> am not a COBOL programmer, so I am sure I am missing parts of the sort
> call, but this is what I was given...
>
> SD  S1-SORT-FILE
>  RECORDING MODE IS F
>  DATA RECORDS ARE S10-SORT-REC
>  RECORD CONTAINS 49 CHARACTERS.
> 01  S10-SORT-REC.
> 05 S20-SORT-KEY.
>10 S20-KEY-LOCATIONPIC X(2).
>10 S20-KEY-MCO PIC X(2).
>10 S20-KEY-PCO PIC X(2).
>10 S20-KEY-TREATY  PIC X(4).
>10 S20-KEY-AS-LINE1PIC 9(3).
>10 S20-KEY-AS-LINE REDEFINES S20-KEY-AS-LINE1
>  PIC 9(2)V9(1).
> 05 S20-BREAK-KEY-R1   REDEFINES S20-SORT-KEY.
>10 S20-TREATY-BREAKPIC X(10).
>10 FILLER  PIC X(3).
> 05 S20-BREAK-KEY-R2   REDEFINES S20-SORT-KEY.
>10 S20-PCO-BREAK   PIC X(6).
>10 FILLER  PIC X(7).
> 05 S20-BREAK-KEY-R3   REDEFINES S20-SORT-KEY.
>10 S20-MCO-BREAK   PIC X(4).
>10 FILLER  PIC X(9).
>05 S30-SORT-BODY.
>   10 S30-LINE-TYPE   PIC 9(1).
>   10 S30-MONTHLY-WRITTEN PIC S9(7)V9(2) COMP-3.
>   10 S30-TO-DATE-WRITTEN PIC S9(7)V9(2) COMP-3.
>   10 S30-NEW-MONTH   PIC S9(7)V9(2) COMP-3.
>   10 S30-THIS-MONTH-EARNED   PIC S9(7)V9(2) COMP-3.
>   10 S30-INFORCE-TO-DATE PIC S9(7)V9(2) COMP-3.
>   10 S30-LA-COMMISSION   PIC S9(7)V9(2) COMP-3.
>   10 S30-TOT-COMMISSION  PIC S9(7)V9(2) COMP-3.
>EJECT
>
> SORT S1-SORT-FILE
> ASCENDING KEY
> S20-SORT-KEY
>
> If anyone could point me in the right direction on this, I would
> appreciate it!
>
> Nathan Pfister
> Systems Programmer
> Donegal Insurance Group
>
> E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance
> Group may contain CONFIDENTIAL and legally protected information. If
> you are not an intended recipient, please do not copy, use or disclose
> this email or its contents to others; and please notify us by calling
> toll free (800) 877-0600 x7880 or by replying to this message, and

Re: Sort Question

[Lizette Koehler]

If possible, post the entire DFSORT output.

There are many elements in the various messages that will help understanding the
issue.  I am interesting in the parms being passed to sort, the rec in/out info
and a few other pieces

The Cobol code not so much

Any changes to the program recently like LRECL or increase in records being
processed?



Also, if you did not know this, you can email the dfsort team directly for
assistance.

dfsorthotl...@us.ibm.com

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Vernooij, Kees (ITOPT1) - KLM
> Sent: Monday, October 02, 2017 6:47 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
> 
> Apparently your sortwk's are that small, that many more are needed.
> 
> Post your sortparameter, displayable with this job:
> //LISTDEF EXEC PGM=ICETOOL
> //TOOLMSG DD SYSOUT=*
> //DFSMSG  DD SYSOUT=*
> //SHOWDEF DD SYSOUT=*
> //TOOLIN  DD *
>   DEFAULTS LIST(SHOWDEF)
> /*
> 
> Post the full DFSORT output and the JCL of the step with the error.
> 
> This info must give a clue.
> 
> Kees.
> 
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> > On Behalf Of Pfister, Nathan
> > Sent: 02 October, 2017 15:40
> > To: IBM-MAIN@LISTSERV.UA.EDU
> > Subject: Sort Question
> >
> > List;
> >
> > I have a question dealing with a DFSORT job which I am hoping to get
> > some guidance on.
> >
> > We have a COBOL program which inconsistently gets a SORT CAPACITY
> > EXCEEDED error.  They've changed their cards several times with
> > several different options to try and get this to work.  Depending on
> > the month sometimes it works and sometimes it doesn't, with their
> > typical OPTION
> > DYNSPC=1024,DYNALLOC=(SYSDA,59)
> >
> > The latest run, we added FILSZ=E6000
> >
> > In every run it seems to exceed the sort capacity.
> > +ICE046A 5 PRSPM175.J05  SORT CAPACITY EXCEEDED - RECORD COUNT
> > 58081219
> >
> > And the following error as well.  Not sure
> >
> > ICE278I 1 59 WORK DATA SETS WERE INSUFFICIENT TO COMPLETE THIS SORT SO
> > 44 ADDITIONAL WERE USED
> >
> > The funny thing is, this has worked with a DYNALLOC of 24...there is
> > nothing else in the system at the time this was running, and plenty of
> > DASD.  Obviously I am overlooking some key piece of information.
> >
> > Here's the COBOL for the Sort, which doesn't mean too much to me as I
> > am not a COBOL programmer, so I am sure I am missing parts of the sort
> > call, but this is what I was given...
> >
> > SD  S1-SORT-FILE
> >  RECORDING MODE IS F
> >  DATA RECORDS ARE S10-SORT-REC
> >  RECORD CONTAINS 49 CHARACTERS.
> > 01  S10-SORT-REC.
> > 05 S20-SORT-KEY.
> >10 S20-KEY-LOCATIONPIC X(2).
> >10 S20-KEY-MCO PIC X(2).
> >10 S20-KEY-PCO PIC X(2).
> >10 S20-KEY-TREATY  PIC X(4).
> >10 S20-KEY-AS-LINE1PIC 9(3).
> >10 S20-KEY-AS-LINE REDEFINES S20-KEY-AS-LINE1
> >  PIC 9(2)V9(1).
> > 05 S20-BREAK-KEY-R1   REDEFINES S20-SORT-KEY.
> >10 S20-TREATY-BREAKPIC X(10).
> >10 FILLER  PIC X(3).
> > 05 S20-BREAK-KEY-R2   REDEFINES S20-SORT-KEY.
> >10 S20-PCO-BREAK   PIC X(6).
> >10 FILLER  PIC X(7).
> > 05 S20-BREAK-KEY-R3   REDEFINES S20-SORT-KEY.
> >10 S20-MCO-BREAK   PIC X(4).
> >10 FILLER  PIC X(9).
> >05 S30-SORT-BODY.
> >   10 S30-LINE-TYPE   PIC 9(1).
> >   10 S30-MONTHLY-WRITTEN PIC S9(7)V9(2) COMP-3.
> >   10 S30-TO-DATE-WRITTEN PIC S9(7)V9(2) COMP-3.
> >   10 S30-NEW-MONTH   PIC S9(7)V9(2) COMP-3.
> >   10 S30-THIS-MONTH-EARNED   PIC S9(7)V9(2) COMP-3.
> >   10 S30-INFORCE-TO-DATE PIC S9(7)V9(2) COMP-3.
> >   10 S30-LA-COMMISSION   PIC S9(7)V9(2) COMP-3.
> >   10 S30-TOT-COMMISSION  PIC S9(7)V9(2) COMP-3.
> >EJECT
> >
> > SORT S1-SORT-FILE
> > ASCENDING KEY
> > S20-SORT-KEY
> >
> > If anyone could point me in the right direction on this, I would
> > appreciate it!
> >
> > Nathan Pfister
> > Systems Programmer
> > Donegal Insurance Group
> >

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Vernooij, Kees (ITOPT1) - KLM]

Apparently your sortwk's are that small, that many more are needed.

Post your sortparameter, displayable with this job:
//LISTDEF EXEC PGM=ICETOOL   
//TOOLMSG DD SYSOUT=*
//DFSMSG  DD SYSOUT=*
//SHOWDEF DD SYSOUT=*
//TOOLIN  DD *   
  DEFAULTS LIST(SHOWDEF) 
/*   

Post the full DFSORT output and the JCL of the step with the error.

This info must give a clue.

Kees.

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Pfister, Nathan
> Sent: 02 October, 2017 15:40
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Sort Question
> 
> List;
> 
> I have a question dealing with a DFSORT job which I am hoping to get
> some guidance on.
> 
> We have a COBOL program which inconsistently gets a SORT CAPACITY
> EXCEEDED error.  They've changed their cards several times with several
> different options to try and get this to work.  Depending on the month
> sometimes it works and sometimes it doesn't, with their typical OPTION
> DYNSPC=1024,DYNALLOC=(SYSDA,59)
> 
> The latest run, we added FILSZ=E6000
> 
> In every run it seems to exceed the sort capacity.
> +ICE046A 5 PRSPM175.J05  SORT CAPACITY EXCEEDED - RECORD COUNT
> 58081219
> 
> And the following error as well.  Not sure
> 
> ICE278I 1 59 WORK DATA SETS WERE INSUFFICIENT TO COMPLETE THIS SORT SO
> 44 ADDITIONAL WERE USED
> 
> The funny thing is, this has worked with a DYNALLOC of 24...there is
> nothing else in the system at the time this was running, and plenty of
> DASD.  Obviously I am overlooking some key piece of information.
> 
> Here's the COBOL for the Sort, which doesn't mean too much to me as I am
> not a COBOL programmer, so I am sure I am missing parts of the sort
> call, but this is what I was given...
> 
> SD  S1-SORT-FILE
>  RECORDING MODE IS F
>  DATA RECORDS ARE S10-SORT-REC
>  RECORD CONTAINS 49 CHARACTERS.
> 01  S10-SORT-REC.
> 05 S20-SORT-KEY.
>10 S20-KEY-LOCATIONPIC X(2).
>10 S20-KEY-MCO PIC X(2).
>10 S20-KEY-PCO PIC X(2).
>10 S20-KEY-TREATY  PIC X(4).
>10 S20-KEY-AS-LINE1PIC 9(3).
>10 S20-KEY-AS-LINE REDEFINES S20-KEY-AS-LINE1
>  PIC 9(2)V9(1).
> 05 S20-BREAK-KEY-R1   REDEFINES S20-SORT-KEY.
>10 S20-TREATY-BREAKPIC X(10).
>10 FILLER  PIC X(3).
> 05 S20-BREAK-KEY-R2   REDEFINES S20-SORT-KEY.
>10 S20-PCO-BREAK   PIC X(6).
>10 FILLER  PIC X(7).
> 05 S20-BREAK-KEY-R3   REDEFINES S20-SORT-KEY.
>10 S20-MCO-BREAK   PIC X(4).
>10 FILLER  PIC X(9).
>05 S30-SORT-BODY.
>   10 S30-LINE-TYPE   PIC 9(1).
>   10 S30-MONTHLY-WRITTEN PIC S9(7)V9(2) COMP-3.
>   10 S30-TO-DATE-WRITTEN PIC S9(7)V9(2) COMP-3.
>   10 S30-NEW-MONTH   PIC S9(7)V9(2) COMP-3.
>   10 S30-THIS-MONTH-EARNED   PIC S9(7)V9(2) COMP-3.
>   10 S30-INFORCE-TO-DATE PIC S9(7)V9(2) COMP-3.
>   10 S30-LA-COMMISSION   PIC S9(7)V9(2) COMP-3.
>   10 S30-TOT-COMMISSION  PIC S9(7)V9(2) COMP-3.
>EJECT
> 
> SORT S1-SORT-FILE
> ASCENDING KEY
> S20-SORT-KEY
> 
> If anyone could point me in the right direction on this, I would
> appreciate it!
> 
> Nathan Pfister
> Systems Programmer
> Donegal Insurance Group
> 
> E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance Group
> may contain CONFIDENTIAL and legally protected information. If you are
> not an intended recipient, please do not copy, use or disclose this
> email or its contents to others; and please notify us by calling toll
> free (800) 877-0600 x7880 or by replying to this message, and then
> delete it from your system. Delivery of this email to an unintended
> recipient is not a waiver of any attorney-client or other applicable
> privilege.
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action rela

Sort Question

[Pfister, Nathan]

List;

I have a question dealing with a DFSORT job which I am hoping to get some 
guidance on.

We have a COBOL program which inconsistently gets a SORT CAPACITY EXCEEDED 
error.  They've changed their cards several times with several different 
options to try and get this to work.  Depending on the month sometimes it works 
and sometimes it doesn't, with their typical OPTION 
DYNSPC=1024,DYNALLOC=(SYSDA,59)

The latest run, we added FILSZ=E6000

In every run it seems to exceed the sort capacity.
+ICE046A 5 PRSPM175.J05  SORT CAPACITY EXCEEDED - RECORD COUNT 58081219

And the following error as well.  Not sure

ICE278I 1 59 WORK DATA SETS WERE INSUFFICIENT TO COMPLETE THIS SORT SO 44 
ADDITIONAL WERE USED

The funny thing is, this has worked with a DYNALLOC of 24...there is nothing 
else in the system at the time this was running, and plenty of DASD.  Obviously 
I am overlooking some key piece of information.

Here's the COBOL for the Sort, which doesn't mean too much to me as I am not a 
COBOL programmer, so I am sure I am missing parts of the sort call, but this is 
what I was given...

SD  S1-SORT-FILE
 RECORDING MODE IS F
 DATA RECORDS ARE S10-SORT-REC
 RECORD CONTAINS 49 CHARACTERS.
01  S10-SORT-REC.
05 S20-SORT-KEY.
   10 S20-KEY-LOCATIONPIC X(2).
   10 S20-KEY-MCO PIC X(2).
   10 S20-KEY-PCO PIC X(2).
   10 S20-KEY-TREATY  PIC X(4).
   10 S20-KEY-AS-LINE1PIC 9(3).
   10 S20-KEY-AS-LINE REDEFINES S20-KEY-AS-LINE1
 PIC 9(2)V9(1).
05 S20-BREAK-KEY-R1   REDEFINES S20-SORT-KEY.
   10 S20-TREATY-BREAKPIC X(10).
   10 FILLER  PIC X(3).
05 S20-BREAK-KEY-R2   REDEFINES S20-SORT-KEY.
   10 S20-PCO-BREAK   PIC X(6).
   10 FILLER  PIC X(7).
05 S20-BREAK-KEY-R3   REDEFINES S20-SORT-KEY.
   10 S20-MCO-BREAK   PIC X(4).
   10 FILLER  PIC X(9).
   05 S30-SORT-BODY.
  10 S30-LINE-TYPE   PIC 9(1).
  10 S30-MONTHLY-WRITTEN PIC S9(7)V9(2) COMP-3.
  10 S30-TO-DATE-WRITTEN PIC S9(7)V9(2) COMP-3.
  10 S30-NEW-MONTH   PIC S9(7)V9(2) COMP-3.
  10 S30-THIS-MONTH-EARNED   PIC S9(7)V9(2) COMP-3.
  10 S30-INFORCE-TO-DATE PIC S9(7)V9(2) COMP-3.
  10 S30-LA-COMMISSION   PIC S9(7)V9(2) COMP-3.
  10 S30-TOT-COMMISSION  PIC S9(7)V9(2) COMP-3.
   EJECT

SORT S1-SORT-FILE
ASCENDING KEY
S20-SORT-KEY

If anyone could point me in the right direction on this, I would appreciate it!

Nathan Pfister
Systems Programmer
Donegal Insurance Group

E-MAIL CONFIDENTIALITY NOTICE: This e-mail from Donegal Insurance Group may 
contain CONFIDENTIAL and legally protected information. If you are not an 
intended recipient, please do not copy, use or disclose this email or its 
contents to others; and please notify us by calling toll free (800) 877-0600 
x7880 or by replying to this message, and then delete it from your system. 
Delivery of this email to an unintended recipient is not a waiver of any 
attorney-client or other applicable privilege.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

>>> RFC 822 allows any displayable USASCII character if it's surrounded by 
quotation  marks.  But phsiii points out that so many sites disallow it 
that 
 it's hardly a concern.

Paul,

Well even though it is NOT a concern, I fixed the job not to depend on the 
delimiters. Just move the name to the end and have the domain up front. By 
doing so we don't have to worry having delimiters or parsing into domain 
data.  So here are the updated control cards

//SYSINDD * 
  INREC IFTHEN=(WHEN=INIT, 
 PARSE=(%01=(ENDBEFR=C'@',FIXLEN=50),   $ Get Field Before @ 
%02=(ENDBEFR=C'>',  $ Get Field Before < 
 ENDBEFR=C',',  $ Get Field Before , 
 ENDBEFR=C' ',FIXLEN=50)),  $ Get Field Before ' ' 
   OVERLAY=(081:%02,TRAN=LTOU,  $ Put Domain AT 081 
131:%01,TRAN=LTOU)),$ Put Name  AT 131 
 
   IFTHEN=(WHEN=INIT, 
  PARSE=(%03=(ABSPOS=131,STARTAFT=C'<', $ Reparse The Name 
 FIXLEN=50)), 
   OVERLAY=(181:%03)),  $ Put Name AT 181 
 
  IFTHEN=(WHEN=INIT, 
   FINDREP=(STARTPOS=181,INOUT=(C'"',C''))),$ REPLACE THE " 
 
  IFTHEN=(WHEN=(181,50,CH,GT,C' '), 
  BUILD=(001,80,$ Original Rec 
 181,50,$ Name 
 081,50)),  $ Domain 
 
  IFTHEN=(WHEN=NONE, 
  BUILD=(001,80,$ Original Rec 
 131,50,$ NAME 
 081,50))   $ DOMAIN 
 
  SORT FIELDS=(081,50,CH,A, $ Name 
   131,50,CH,A),EQUALS  $ Domain 
/* 

Thanks,
Kolusu
DFSORT Development
IBM Corporation

IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on 
07/19/2017 09:40:11 AM:

> From: Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 07/19/2017 09:40 AM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>
> 
> On Wed, 19 Jul 2017 08:57:49 -0700, Sri h Kolusu wrote:
> 
> >>> #$%& is indeed delimiter, any character set, anything which cannot 
be 
> >found in real email address.
> > 
> RFC 822 allows any displayable USASCII character if it's surrounded 
> by quotation
> marks.  But phsiii points out that so many sites disallow it that 
> it's hardly a concern.
> 
> -- gil
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Wed, 19 Jul 2017 08:57:49 -0700, Sri h Kolusu wrote:

>>> #$%& is indeed delimiter, any character set, anything which cannot be 
>found in real email address.
> 
RFC 822 allows any displayable USASCII character if it's surrounded by quotation
marks.  But phsiii points out that so many sites disallow it that it's hardly a 
concern.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

>> #$%& is indeed delimiter, any character set, anything which cannot be 
found in real email address.

R.S,

Yes that is indeed my delimiter that I added in the earlier on the 1st 
IFTHEN statement, where I was converting to upper case.  I added that 
delimiter as stopper to stop PARSE from getting into the domain data when 
we are re-parsing the name once again. May be I should have added more 
detail comments but then it would overshadow the actual control cards. So 
I kept the comments to be short. 

>>> Sri, you're great guy, I appreciate your contribution very much!

Thank You


Thanks,
Kolusu

IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on 
07/19/2017 06:37:33 AM:

> From: "R.S." <r.skoru...@bremultibank.com.pl>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 07/19/2017 06:38 AM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>
> 
> W dniu 2017-07-19 o 15:11, R.S. pisze:
> > W dniu 2017-07-17 o 17:33, Sri h Kolusu pisze:
> >>>> I see the following room for improvement: 1. Do NOT limit domain 
field
> >> to first dot. Just take it as whole string.
> >> Example of sorted domains
> >>
> >> R.S
> >>
> >> My earlier job was just an untested sample that I wrote on the 
> >> weekend to
> >> meet Edward's requirement.  Here is a fully tested version and it 
takes
> >> into consideration all the points raised here so far.  Thanks paul 
for
> >> sending in some sample data. It also handles the lower case and upper
> >> case. We only convert the parsed values to upper case and leave the
> >> original record as is.
> >>
> >> I did not code the OUTREC statement to remove the parsed fields as I
> >> wanted to show how the values are parsed. If the job met all your
> >> requirements then simply add OUTREC BUILD=(1,80)
> >>
> >> //STEP0100 EXEC PGM=SORT
> >> //SYSOUT   DD SYSOUT=*
> >> //SORTIN   DD *
> >> John Sr Doe <john...@us.ibm.com>,
> >> SORT EXPERT <john...@us.ibm.com>,
> >> john...@us.ibm.com,
> >> <john...@us.ibm.com>,
> >> <"JOHNDOE"@US.IBM.COM>,
> >> (MUST BE EQUIVALENT) <"JOHNDOE"@US.IBM.COM>,
> >> (OPTIONALLY DISTINCT) <"JOHNDOE"@US.IBM.COM>,
> >> (MAY BE REJECTED BY IBM) <"JOHNDOE+FOLDER"@US.IBM.COM>
> >> x...@bbb.com
> >> aa...@aaa.com
> >> b...@aa.com
> >> n...@example.com
> >> a.b@email.com
> >> exampleem...@domain.in
> >> walt.farr...@yahoo.co.uk
> >> a...@domain.com
> >> a...@domain.com.pl
> >> a...@domain.pl
> >> a...@domain.de
> >> a...@domain.in
> >> a...@domain.pl
> >> //SORTOUT  DD SYSOUT=*
> >>INREC IFTHEN=(WHEN=INIT,
> >>   PARSE=(%01=(ENDBEFR=C'@',FIXLEN=50),   $ GET FIELD BEFORE @
> >>  %02=(ENDBEFR=C'>',  $ GET FIELD BEFORE <
> >>   ENDBEFR=C',',  $ GET FIELD BEFORE ,
> >>   ENDBEFR=C' ',FIXLEN=50)),  $ GET FIELD BEFORE ' 
'
> >> OVERLAY=(081:%01,TRAN=LTOU,  $ PUT NAME AT 81
> >>  C'#$%&',$ ADD SPL CHAR #$%&
> >>  135:%02,TRAN=LTOU)),$ PUT DOMAIN AT 135
> >> IFTHEN=(WHEN=INIT,
> >>PARSE=(%03=(ABSPOS=81,STARTAFT=C'<',  $ REPARSE THE NAME
> >>   ENDBEFR=C'#$%&',   $ SPL CHAR AS 
STOPPER
> >>   FIXLEN=50)),
> >> OVERLAY=(185:%03)),  $ PUT NAME AT 185
> >>  IFTHEN=(WHEN=INIT,
> >> FINDREP=(STARTPOS=181,INOUT=(C'"',C''))),$ REPLACE THE "
> >>IFTHEN=(WHEN=(185,50,CH,GT,C' '),
> >>BUILD=(001,80,$ ORIGINAL REC
> >>   185,50,$ NAME
> >>   135,50)),  $ DOMAIN
> >>  IFTHEN=(WHEN=NONE,
> >>BUILD=(001,80,$ ORIGINAL REC
> >>   081,50,$ NAME
> >>   135,50))   $ DOMAIN
> >>  SORT FIELDS=(081,50,CH,A, $ NAME
> >> 131,50,CH,A),EQUALS  $ DOMAIN
> >> /*
> >>
> >> Thanks,
> >> Kolusu
> >> DFSORT Development
> >> IBM Corporation
>

Re: Sort Question

[R.S.]

W dniu 2017-07-17 o 17:33, Sri h Kolusu pisze:

I see the following room for improvement: 1. Do NOT limit domain field

to first dot. Just take it as whole string.
Example of sorted domains

R.S

My earlier job was just an untested sample that I wrote on the weekend to
meet Edward's requirement.  Here is a fully tested version and it takes
into consideration all the points raised here so far.  Thanks paul for
sending in some sample data. It also handles the lower case and upper
case. We only convert the parsed values to upper case and leave the
original record as is.

I did not code the OUTREC statement to remove the parsed fields as I
wanted to show how the values are parsed. If the job met all your
requirements then simply add OUTREC BUILD=(1,80)

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
John Sr Doe ,
SORT EXPERT ,
john...@us.ibm.com,
,
<"JOHNDOE"@US.IBM.COM>,
(MUST BE EQUIVALENT) <"JOHNDOE"@US.IBM.COM>,
(OPTIONALLY DISTINCT) <"JOHNDOE"@US.IBM.COM>,
(MAY BE REJECTED BY IBM) <"JOHNDOE+FOLDER"@US.IBM.COM>
x...@bbb.com
aa...@aaa.com
b...@aa.com
n...@example.com
a.b@email.com
exampleem...@domain.in
walt.farr...@yahoo.co.uk
a...@domain.com
a...@domain.com.pl
a...@domain.pl
a...@domain.de
a...@domain.in
a...@domain.pl
//SORTOUT  DD SYSOUT=*
   INREC IFTHEN=(WHEN=INIT,
  PARSE=(%01=(ENDBEFR=C'@',FIXLEN=50),   $ GET FIELD BEFORE @
 %02=(ENDBEFR=C'>',  $ GET FIELD BEFORE <
  ENDBEFR=C',',  $ GET FIELD BEFORE ,
  ENDBEFR=C' ',FIXLEN=50)),  $ GET FIELD BEFORE ' '
OVERLAY=(081:%01,TRAN=LTOU,  $ PUT NAME AT 81
 C'#$%&',$ ADD SPL CHAR #$%&
 135:%02,TRAN=LTOU)),$ PUT DOMAIN AT 135
IFTHEN=(WHEN=INIT,
   PARSE=(%03=(ABSPOS=81,STARTAFT=C'<',  $ REPARSE THE NAME
  ENDBEFR=C'#$%&',   $ SPL CHAR AS STOPPER
  FIXLEN=50)),
OVERLAY=(185:%03)),  $ PUT NAME AT 185
  
   IFTHEN=(WHEN=INIT,

FINDREP=(STARTPOS=181,INOUT=(C'"',C''))),$ REPLACE THE "
  
  
   IFTHEN=(WHEN=(185,50,CH,GT,C' '),

   BUILD=(001,80,$ ORIGINAL REC
  185,50,$ NAME
  135,50)),  $ DOMAIN
  
   IFTHEN=(WHEN=NONE,

   BUILD=(001,80,$ ORIGINAL REC
  081,50,$ NAME
  135,50))   $ DOMAIN
  
   SORT FIELDS=(081,50,CH,A, $ NAME

131,50,CH,A),EQUALS  $ DOMAIN
/*
  


Thanks,
Kolusu
DFSORT Development
IBM Corporation


OK, I give up.
The job is obvoiusly working OK, however I can't fully understand its 
logic.


My understanding:
** first IFTHEN + OVERLAY put string before @ (with all the garbage like 
"John Sr Doe <") and string before end of domain (which can be space or 
">" or comma). First string is placed in col 81, the second in col 135.

What is "#$%&" ? is kind of delimiter?

** the second IFTHEN parses "name" field again to remove "<". What about 
records without "<>" ?


** third IFTHEN simply removes " character, but why it starts from 181, 
while name was put on 185?


** fourth IFTHEN chooses records with non-empty field 185,50 and builds 
record using original record (1,80), name (185,50) and domain (135,50).


** fifth IFTHEN takes rest of records (means empty 185,50) and builds 
record using original record (1,80) "raw" name (81,50) and domain (131,50).


--
Radoslaw Skorupka
Lodz, Poland




==


   --
Treść tej wiadomości może zawierać informacje prawnie chronione Banku 
przeznaczone wyłącznie do użytku służbowego adresata. Odbiorcą może być jedynie 
jej adresat z wyłączeniem dostępu osób trzecich. Jeżeli nie jesteś adresatem 
niniejszej wiadomości lub pracownikiem upoważnionym do jej przekazania 
adresatowi, informujemy, że jej rozpowszechnianie, kopiowanie, rozprowadzanie 
lub inne działanie o podobnym charakterze jest prawnie zabronione i może być 
karalne. Jeżeli otrzymałeś tę wiadomość omyłkowo, prosimy niezwłocznie 
zawiadomić nadawcę wysyłając odpowiedź oraz trwale usunąć tę wiadomość 
włączając w to wszelkie jej kopie wydrukowane lub zapisane na dysku.

This e-mail may contain legally privileged information of the Bank and is 
intended solely for business use of the addressee. This e-mail may only be 
received by the addressee and may not be disclosed to any third parties. If you 
are not the intended addressee of this e-mail or the employee authorized to 
forward it to the addressee, be advised that any dissemination, copying, 
distribution or any other similar activity is legally prohibited and may be 
punishable. If you received this e-mail by mistake 

Re: Sort Question

[Edward Gould]

> On Jul 18, 2017, at 1:23 AM, Paul Gilmartin 
> <000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
> 
> How challenging were his data?  In the easiest case, no "phrase" and
> no quotation marks.
> 
> — gil

Gil:
I will give him somewhat the benefit of the doubt the hardcopy I gave him 
wasn’t the greatest.

Ed 


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Mon, 17 Jul 2017 23:34:28 -0500, Edward Gould wrote:
>
>The user says he has about 20 million records. It is some sort of accumulated 
>history file. Don’t know any other details.
>I gave him the IBM solution and told him to come back if he had any problems.
>I was watching SDSF and a job poped up with his ID, so I decided to snoop a 
>little. He had difficulties with the sort control statements and it took him 
>several times to get them right.
>When his job finally got going, there were 25 million records.
>I am waiting too hear if the output was in the desired format.
>
How challenging were his data?  In the easiest case, no "phrase" and
no quotation marks.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Lizette Koehler]

Wow - that dooms Excel.  I think a million lines might be the max for it

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Edward Gould
> Sent: Monday, July 17, 2017 9:34 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
> 
> > On Jul 15, 2017, at 3:43 PM, Lizette Koehler <stars...@mindspring.com>
> wrote:
> >
> > Does this need to be a mainframe solution?
> >
> > How many records are involved?
> >
> > If not a lot of data, I load this type of data to Excel.
> >
> > The using the DATA tab, I then highlight the column and use the TEXT
> > TO COLUMS function to break on the @
> >
> > Do my sorts, then put it back together (using CONCAT function), or
> > leave in excel for further filtering.
> >
> > Lizette
> >
> 
> Lizette,
> 
> The user says he has about 20 million records. It is some sort of accumulated
> history file. Don’t know any other details.
> I gave him the IBM solution and told him to come back if he had any problems.
> I was watching SDSF and a job poped up with his ID, so I decided to snoop a
> little. He had difficulties with the sort control statements and it took him
> several times to get them right.
> When his job finally got going, there were 25 million records.
> I am waiting too hear if the output was in the desired format.
> 
> Ed

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Edward Gould]

> On Jul 15, 2017, at 3:43 PM, Lizette Koehler  wrote:
> 
> Does this need to be a mainframe solution?
> 
> How many records are involved?
> 
> If not a lot of data, I load this type of data to Excel.
> 
> The using the DATA tab, I then highlight the column and use the TEXT TO COLUMS
> function to break on the @
> 
> Do my sorts, then put it back together (using CONCAT function), or leave in
> excel for further filtering.
> 
> Lizette
> 

Lizette,

The user says he has about 20 million records. It is some sort of accumulated 
history file. Don’t know any other details.
I gave him the IBM solution and told him to come back if he had any problems.
I was watching SDSF and a job poped up with his ID, so I decided to snoop a 
little. He had difficulties with the sort control statements and it took him 
several times to get them right.
When his job finally got going, there were 25 million records.
I am waiting too hear if the output was in the desired format.

Ed
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Mon, 17 Jul 2017 12:33:31 -0400, Tony Harminc wrote:
>
>... there is currently no other Tony in the email list at
>this domain, so Tony is enough to reach me. That will surely change
>one day, but what will the algorithm be? Reject, ignore, stick to the
>existing rule and send to me...?
>
Closely related to the curse of autocompletion.

>But any RFC-compliant mail *sender* must permit the + character in the
>local-part. This is a pet peeve. Many web sites have a similer piece
>of bogus Javascript that rejects all kinds of perfectly valid
>addresses. I like to use the + convention, but I'd say around 60% of
>sites I deal with reject it. The worst silently ignore the +, or
>replace it with a blank, and let the next downstream component fail on
>it, also perhaps silently. I have had helpdesks tell me I must change
>my email address in order to deal with their company. Would they do
>the same if their phone system had a bug such that my area code wasn't
>accepted?*
> 
Grrr.   Repeal and Replace Postel's Law.

>* "...switch programming error in Toronto". ...
>
Had one last week.  My phone provider is CenturyLink.  My caller ID
was appearing, only to Comcast subscribers, as "Boulder Community
Hospital".  Unnerving to see late at night.  Called Comcast support,
since problem appeared peculiar to Comcast.  "Vince" was adamant
that problem was with CenturyLink "outbound caller ID", and I needed
to call CenturyLink.  Wouldn't even give me an SR number since I
don't have a Comcast account.  "Michelle" at CenturyLink was more
helpful.  She agreed to contact Comcast, giving me an SR number
that looked just like a 1-800... phone nunber and put me on hold.
After a few minutes of background music, Comcast picked up.  Oops!
But after I navigated the voice menu, "Shea" was more helpful.  With
more holds, he came back snd reported a "switch programming error"
and submitted a request to fix it, saying that should have been
CenturyLink's job, and that "Vince" should have been able to give me
the same assistance.  Fixed in 12 hours.  Gasp!

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Tony Harminc]

On 16 July 2017 at 10:28, Paul Gilmartin
<000433f07816-dmarc-requ...@listserv.ua.edu> wrote:

> (From RFC 822:
>  addr-spec   =  local-part "@" domain; global address )
...
> If the local-part contains special characters, it must be surrounded by
> quotation marks.  If there are no special characters, the quotation
> marks are optional.

But note that many non-alphanumeric characters are not "special" in
this context, and so do not need to be quoted. For example + and & are
not special, so
Bob+Ted@example.com
is valid.

> Mappings may make domains equivalent.  For example, my domain
> may be written as "AIM.com" or "AOL.Com"

Mappings may also make local-parts equivalent. Gmail, for example,
ignores embedded periods when processing inbound mail. My employer's
inbound email handling applies some kind of search to the local-part.
As it happens, there is currently no other Tony in the email list at
this domain, so Tony is enough to reach me. That will surely change
one day, but what will the algorithm be? Reject, ignore, stick to the
existing rule and send to me...?

> Some servers (not all) accept userID"+"subaccount as a local-part,
> delivering mail to userID and allowing rules in userID's MUA to
> sort messages into folders.

But any RFC-compliant mail *sender* must permit the + character in the
local-part. This is a pet peeve. Many web sites have a similer piece
of bogus Javascript that rejects all kinds of perfectly valid
addresses. I like to use the + convention, but I'd say around 60% of
sites I deal with reject it. The worst silently ignore the +, or
replace it with a blank, and let the next downstream component fail on
it, also perhaps silently. I have had helpdesks tell me I must change
my email address in order to deal with their company. Would they do
the same if their phone system had a bug such that my area code wasn't
accepted?*

Most of the big vendors' mail products get this stuff right, but even
a few of them have quirks. In my experience Gmail is near perfect, MS
Outlook almost as good, IBM/Lotus Notes not so great, and


> ... all challenges for DFSORT; some not algorithmically solvable.

The syntactic validity of an email address is algorithmically
solvable, but the processing and policies of various recipient servers
and organizations is of course, not. Hence no amount of algorithmic
normalization can reliably tell you if two email address are
equivalent or not.

Tony H.

* Actually I had this problem around ten years ago. Trying to call a
number in the UK from a landline here in Canada. "Cannot complete your
call as dialled" recording. The operator (does anyone remember the
notion of telephone operators?) tried with the same results, and
everyone told me the UK number was invalid. I got someone else in the
UK to call it, and they got through. Eventually persuaded my local
operator to call the UK operator and have them try it. Went through
with no trouble. I opened a ticket with repair service (which they
didn't want to accept), and eventually they got back to me with "a
switch programming error in Toronto". How do you call an operator in
2017...?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

>>I see the following room for improvement: 1. Do NOT limit domain field 
to first dot. Just take it as whole string.
Example of sorted domains

R.S

My earlier job was just an untested sample that I wrote on the weekend to 
meet Edward's requirement.  Here is a fully tested version and it takes 
into consideration all the points raised here so far.  Thanks paul for 
sending in some sample data. It also handles the lower case and upper 
case. We only convert the parsed values to upper case and leave the 
original record as is.

I did not code the OUTREC statement to remove the parsed fields as I 
wanted to show how the values are parsed. If the job met all your 
requirements then simply add OUTREC BUILD=(1,80)

//STEP0100 EXEC PGM=SORT 
//SYSOUT   DD SYSOUT=* 
//SORTIN   DD * 
John Sr Doe <john...@us.ibm.com>, 
SORT EXPERT <john...@us.ibm.com>, 
john...@us.ibm.com, 
<john...@us.ibm.com>, 
<"JOHNDOE"@US.IBM.COM>, 
(MUST BE EQUIVALENT) <"JOHNDOE"@US.IBM.COM>, 
(OPTIONALLY DISTINCT) <"JOHNDOE"@US.IBM.COM>, 
(MAY BE REJECTED BY IBM) <"JOHNDOE+FOLDER"@US.IBM.COM> 
x...@bbb.com 
aa...@aaa.com 
b...@aa.com 
n...@example.com 
a.b@email.com 
exampleem...@domain.in 
walt.farr...@yahoo.co.uk 
a...@domain.com 
a...@domain.com.pl 
a...@domain.pl 
a...@domain.de 
a...@domain.in 
a...@domain.pl 
//SORTOUT  DD SYSOUT=* 
  INREC IFTHEN=(WHEN=INIT, 
 PARSE=(%01=(ENDBEFR=C'@',FIXLEN=50),   $ GET FIELD BEFORE @ 
%02=(ENDBEFR=C'>',  $ GET FIELD BEFORE < 
 ENDBEFR=C',',  $ GET FIELD BEFORE , 
 ENDBEFR=C' ',FIXLEN=50)),  $ GET FIELD BEFORE ' '
   OVERLAY=(081:%01,TRAN=LTOU,  $ PUT NAME AT 81 
C'#$%&',$ ADD SPL CHAR #$%& 
135:%02,TRAN=LTOU)),$ PUT DOMAIN AT 135 
   IFTHEN=(WHEN=INIT, 
  PARSE=(%03=(ABSPOS=81,STARTAFT=C'<',  $ REPARSE THE NAME 
 ENDBEFR=C'#$%&',   $ SPL CHAR AS STOPPER 
 FIXLEN=50)), 
   OVERLAY=(185:%03)),  $ PUT NAME AT 185 
 
  IFTHEN=(WHEN=INIT, 
   FINDREP=(STARTPOS=181,INOUT=(C'"',C''))),$ REPLACE THE " 
 
 
  IFTHEN=(WHEN=(185,50,CH,GT,C' '), 
  BUILD=(001,80,$ ORIGINAL REC 
 185,50,$ NAME 
 135,50)),  $ DOMAIN 
 
  IFTHEN=(WHEN=NONE, 
  BUILD=(001,80,$ ORIGINAL REC 
 081,50,$ NAME 
 135,50))   $ DOMAIN 
 
  SORT FIELDS=(081,50,CH,A, $ NAME 
   131,50,CH,A),EQUALS  $ DOMAIN 
/* 
 

Thanks,
Kolusu
DFSORT Development
IBM Corporation



From:   "R.S." <r.skoru...@bremultibank.com.pl>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   07/17/2017 05:00 AM
Subject:Re: Sort Question
Sent by:IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>



I see the following room for improvement:
1. Do NOT limit domain field to first dot. Just take it as whole string.
Example of sorted domains
a...@domain.com
a...@domain.com.pl
a...@domain.de
a...@domain.in
a...@domain.pl
How to change "%02=(ENDBEFR=C'.',FIXLEN=30)" to something like "%02=rest 
of record" ?
My idea is to put the following:
%02=(ENDBEFR=X'40',FIXLEN=30)   *just check for space after the record

2. Uppercase and lowercase sorted in same order. For simplicity it can 
also convert UTOL or LTOU.
I think, for small datasets it is reasonable to use pre-step with UTOL 
(first step converts UTOL, second do the main sort).

3. (next level of complexity). Provide "exceptions" for multiple-domain 
companies
Example:
a...@bremultibank.com.pl
a...@mbank.pl
a...@mbank.com.pl
are equal.
I have no idea here :-(

Regards
-- 
Radoslaw Skorupka
Lodz, Poland






W dniu 2017-07-16 o 16:38, Sri h Kolusu pisze:
>>> ... all challenges for DFSORT; some not algorithmically solvable.
> Paul,
> '
> Can you send me a sample of data with all the challenging scenarios and 
I
> will *try* to see if I can solve it. For a fully tested solution it 
would
> be only on Monday.
>
> Thanks,
> Kolusu
>
> IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
> 07/16/2017 07:28:19 AM:
>
>> From: Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Date: 07/16/2017 07:28 AM
>> Subject: Re: Sort Question
>> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>
>>
>> On Sat, 15 Jul 2017 14:18:41 -0500, Edward Gould wrote:
>>
>>> One of my users asked me this question and I am not quite clear on
>> how to respond.
>&g

Sorting email addresses (was: Sort Question)

[Paul Gilmartin]

On 2017-07-17, at 05:58, R.S. wrote:
> 
> 2. Uppercase and lowercase sorted in same order. ...
>  
By Internet standard, the domain is case-insensitive.  RFC 822 et al.
leave interpretation of the local-part entirely to the target site,
which is free to treat it as either case-sensitive or insensitive.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Sorting email addresses (was: Sort Question)

[Paul Gilmartin]

(Subject: changed to be more relevant)

On Sun, 16 Jul 2017 07:38:36 -0700, Sri h Kolusu wrote:
>
>Can you send me a sample of data with all the challenging scenarios and I 
>will *try* to see if I can solve it. For a fully tested solution it would 
>be only on Monday.
> 
This page has a few more samples:

http://codefool.tumblr.com/post/15288874550/list-of-valid-and-invalid-email-addresses

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[R.S.]

I see the following room for improvement:
1. Do NOT limit domain field to first dot. Just take it as whole string.
Example of sorted domains
a...@domain.com
a...@domain.com.pl
a...@domain.de
a...@domain.in
a...@domain.pl
How to change "%02=(ENDBEFR=C'.',FIXLEN=30)" to something like "%02=rest 
of record" ?

My idea is to put the following:
%02=(ENDBEFR=X'40',FIXLEN=30)   *just check for space after the record

2. Uppercase and lowercase sorted in same order. For simplicity it can 
also convert UTOL or LTOU.
I think, for small datasets it is reasonable to use pre-step with UTOL 
(first step converts UTOL, second do the main sort).


3. (next level of complexity). Provide "exceptions" for multiple-domain 
companies

Example:
a...@bremultibank.com.pl
a...@mbank.pl
a...@mbank.com.pl
are equal.
I have no idea here :-(

Regards
--
Radoslaw Skorupka
Lodz, Poland






W dniu 2017-07-16 o 16:38, Sri h Kolusu pisze:

... all challenges for DFSORT; some not algorithmically solvable.

Paul,
'
Can you send me a sample of data with all the challenging scenarios and I
will *try* to see if I can solve it. For a fully tested solution it would
be only on Monday.

Thanks,
Kolusu

IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
07/16/2017 07:28:19 AM:


From: Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date: 07/16/2017 07:28 AM
Subject: Re: Sort Question
Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>

On Sat, 15 Jul 2017 14:18:41 -0500, Edward Gould wrote:


One of my users asked me this question and I am not quite clear on

how to respond.

Given an email address aaa...@bbb.com <mailto:aaa...@bbb.com> (example)

He just wants to sort on the field before the @ and then for a

secondary sort the bbb field. Both fields are variable in length
(not sure what the max is).

Any suggestions on how to do this? I have tried RTFM but maybe I am

not looking at the question correctly .
(From RFC 822:
  addr-spec   =  local-part "@" domain; global address )

The domain is specified (Internet standard) to be case-insensitive.

The interpretation of the local-part is left entirely up to the server,
which may treat it as case-sensitive or insensitive.

If the local-part contains special characters, it must be surrounded by
quotation marks.  If there are no special characters, the quotation
marks are optional.

Mappings may make domains equivalent.  For example, my domain
may be written as "AIM.com" or "AOL.Com"

Some servers (not all) accept userID"+"subaccount as a local-part,
delivering mail to userID and allowing rules in userID's MUA to
sort messages into folders.

And a comment may be profided as a "phrase" in "mailbox"

  mailbox =  addr-spec; simple address
  /  phrase route-addr; name & addr-spec

  route-addr  =  "<" [route] addr-spec ">"

... all challenges for DFSORT; some not algorithmically solvable.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



==


   --
Treść tej wiadomości może zawierać informacje prawnie chronione Banku 
przeznaczone wyłącznie do użytku służbowego adresata. Odbiorcą może być jedynie 
jej adresat z wyłączeniem dostępu osób trzecich. Jeżeli nie jesteś adresatem 
niniejszej wiadomości lub pracownikiem upoważnionym do jej przekazania 
adresatowi, informujemy, że jej rozpowszechnianie, kopiowanie, rozprowadzanie 
lub inne działanie o podobnym charakterze jest prawnie zabronione i może być 
karalne. Jeżeli otrzymałeś tę wiadomość omyłkowo, prosimy niezwłocznie 
zawiadomić nadawcę wysyłając odpowiedź oraz trwale usunąć tę wiadomość 
włączając w to wszelkie jej kopie wydrukowane lub zapisane na dysku.

This e-mail may contain legally privileged information of the Bank and is 
intended solely for business use of the addressee. This e-mail may only be 
received by the addressee and may not be disclosed to any third parties. If you 
are not the intended addressee of this e-mail or the employee authorized to 
forward it to the addressee, be advised that any dissemination, copying, 
distribution or any other similar activity is legally prohibited and may be 
punishable. If you received this e-mail by mistake please advise the sender 
immediately by using the reply facility in your e-mail software and delete 
permanently this e-mail including any copies of it either printed or saved to 
hard drive.

mBank S.A. z siedzibą w Warszawie, ul. Senatorska 18, 00-950 Warszawa, 
www.mBank.pl, e-mail: kont...@mbank.plsąd Rejonowy dla m. st. Warszawy XII 
Wydział Gospodarczy Krajowego Rejes

Re: Sort Question

[Paul Gilmartin]

On Sun, 16 Jul 2017 07:38:36 -0700, Sri h Kolusu wrote:

>>>... all challenges for DFSORT; some not algorithmically solvable.
>'
>Can you send me a sample of data with all the challenging scenarios and I 
>will *try* to see if I can solve it. For a fully tested solution it would 
>be only on Monday.
> 
Sent a few off-list; certainly not all.

Hmmm.  Does z/OS (or another OS) allow "@" in user IDs?  Then:
To: Nuisance User <"TSO@ID"@Example.com>

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

>>... all challenges for DFSORT; some not algorithmically solvable.

Paul,
'
Can you send me a sample of data with all the challenging scenarios and I 
will *try* to see if I can solve it. For a fully tested solution it would 
be only on Monday.

Thanks,
Kolusu

IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on 
07/16/2017 07:28:19 AM:

> From: Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 07/16/2017 07:28 AM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>
> 
> On Sat, 15 Jul 2017 14:18:41 -0500, Edward Gould wrote:
> 
> >One of my users asked me this question and I am not quite clear on 
> how to respond.
> >Given an email address aaa...@bbb.com <mailto:aaa...@bbb.com> (example)
> >
> >He just wants to sort on the field before the @ and then for a 
> secondary sort the bbb field. Both fields are variable in length 
> (not sure what the max is).
> >Any suggestions on how to do this? I have tried RTFM but maybe I am
> not looking at the question correctly .
> >
> (From RFC 822:
>  addr-spec   =  local-part "@" domain; global address )
> 
> The domain is specified (Internet standard) to be case-insensitive.
> 
> The interpretation of the local-part is left entirely up to the server,
> which may treat it as case-sensitive or insensitive.
> 
> If the local-part contains special characters, it must be surrounded by
> quotation marks.  If there are no special characters, the quotation
> marks are optional.
> 
> Mappings may make domains equivalent.  For example, my domain
> may be written as "AIM.com" or "AOL.Com"
> 
> Some servers (not all) accept userID"+"subaccount as a local-part,
> delivering mail to userID and allowing rules in userID's MUA to
> sort messages into folders.
> 
> And a comment may be profided as a "phrase" in "mailbox"
> 
>  mailbox =  addr-spec; simple address
>  /  phrase route-addr; name & addr-spec
> 
>  route-addr  =  "<" [route] addr-spec ">"
> 
> ... all challenges for DFSORT; some not algorithmically solvable.
> 
> -- gil
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

>>>Thanks!… I would never have gotten that without a lot of testing. Also 
I did not know the first “field” was limited to 30. I learned several 
things today. 

Edward,

As retired mainframer pointed out, it was MY assumption about the length 
being 30. You can have whatever you want. Just change the FIXLEN=30 to 
whatever you want and change the positions of OVERLAY and SORT fields 
according to your desired length.

>>Also, the first part of the address can contain "." characters, so the 
second parse will not work correctly in all cases. 

Walt,

Not really. It does not matter as to how many "." you have before "@", the 
first field parse will ignore the ".". So as long as the string as has "@" 
character the first parse will be active and only the contents AFTER the 
"@" will be parsed by the 2nd parse field for the domain.

For example if you had a.b@email.com the first parse field for "@" 
will get the entire string a.b.c.d and the 2nd parse field for "." will 
get the email

However your scenario for "." is true for domain. As some email address 
may contain "." for domain. like for example walt.farr...@yahoo.co.uk

In the above case the name will be parsed as walt.farrell and the domain 
is ONLY parsed to yahoo instead of yahoo.co.uk 

So if OP's has various combos of records with multiple characters of "." 
in the domain name, then he needs to account for them which isn't that 
hard given that he had a template to start with. I just coded it on a 
saturday night with a sample data shown and it is an untested version.

Thanks,
Kolusu

IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on 
07/16/2017 06:33:44 AM:

> From: Walt Farrell <walt.farr...@gmail.com>
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 07/16/2017 06:34 AM
> Subject: Re: Sort Question
> Sent by: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>
> 
> On Sat, 15 Jul 2017 22:52:47 -0500, Edward Gould 
> <edgould1...@comcast.net> wrote:
> 
> >> On Jul 15, 2017, at 10:39 PM, Sri h Kolusu <skol...@us.ibm.com> 
wrote:
> >> 
> >> Edward, 
> >> 
> >> Here is a  DFSORT JCL which will give you the desired results. I 
assumed 
> >> that your input file is FB and 80 bytes in length. I also assumed 
that 
> >> your name and domain names are each 30 bytes in length.
> >> 
> >> //STEP0100 EXEC PGM=SORT 
> >> //SYSOUT   DD SYSOUT=* 
> >> //SORTIN   DD * 
> >> x...@bbb.com <mailto:x...@bbb.com> 
> >> aa...@aaa.com <mailto:aa...@aaa.com> 
> >> b...@aa.com <mailto:b...@aa.com> 
> >> n...@example.com <mailto:n...@example.com> 
> >> exampleem...@domain.in <mailto:exampleem...@domain.in> 
> >> //SORTOUT  DD SYSOUT=* 
> >>  INREC PARSE=(%01=(ENDBEFR=C'@',FIXLEN=30),   $ GET FIELD BEFORE @ 
> >>   %02=(ENDBEFR=C'.',FIXLEN=30)),  $ GETFIELD BEFORE . 
> >>   OVERLAY=(081:%01,   $ PUT NAME AT 81 
> >>111:%02)   $ PUT DOMAIN AT 111 
> >> 
> >>  SORT FIELDS=(081,30,CH,A,$ NAME 
> >>   111,30,CH,A),EQUALS $ DOMAIN 
> >> 
> >>  OUTREC BUILD=(1,80)  $ REMOVE TEMP FIELDS 
> >> /* 
> >> 
> >> The output from this field is
> >> 
> >> aa...@aaa.com <mailto:aa...@aaa.com> 
> >> b...@aa.com <mailto:b...@aa.com> 
> >> exampleem...@domain.in <mailto:exampleem...@domain.in> 
> >> n...@example.com <mailto:n...@example.com> 
> >> x...@bbb.com <mailto:x...@bbb.com> 
> >> 
> >> Further if you have questions please let me know
> >> 
> >> Thanks,
> >> Kolusu
> >> DFSORT Development
> >> IBM Corporation
> >> 
> >
> >Thanks!… I would never have gotten that without a lot of testing. 
> Also I did not know the first “field” was limited to 30. I learned 
> >several things today.
> 
> The part of the email address before the @ is NOT limited to 30 
> characters. Nor is the part after the @. Also, the first part of the
> address can contain "." characters, so the second parse will not 
> work correctly in all cases. 
> 
> If you can guarantee that the  part will always
> be present, then it might be safer to parse that part, looking for 
> the text between "". But in
> any case you will risk truncating the part before the @ unless you 
> allow for the possibility of it being much longer.
> 
> -- 
> Walt
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Paul Gilmartin]

On Sat, 15 Jul 2017 14:18:41 -0500, Edward Gould wrote:

>One of my users asked me this question and I am not quite clear on how to 
>respond.
>Given an email address aaa...@bbb.com  (example)
>
>He just wants to sort on the field before the @ and then for a secondary sort 
>the bbb field. Both fields are variable in length (not sure what the max is).
>Any suggestions on how to do this? I have tried RTFM but maybe I am not 
>looking at the question correctly .
>
(From RFC 822:
 addr-spec   =  local-part "@" domain; global address )

The domain is specified (Internet standard) to be case-insensitive.

The interpretation of the local-part is left entirely up to the server,
which may treat it as case-sensitive or insensitive.

If the local-part contains special characters, it must be surrounded by
quotation marks.  If there are no special characters, the quotation
marks are optional.

Mappings may make domains equivalent.  For example, my domain
may be written as "AIM.com" or "AOL.Com"

Some servers (not all) accept userID"+"subaccount as a local-part,
delivering mail to userID and allowing rules in userID's MUA to
sort messages into folders.

And a comment may be profided as a "phrase" in "mailbox"

 mailbox =  addr-spec; simple address
 /  phrase route-addr; name & addr-spec

 route-addr  =  "<" [route] addr-spec ">"

... all challenges for DFSORT; some not algorithmically solvable.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Walt Farrell]

On Sat, 15 Jul 2017 22:52:47 -0500, Edward Gould  
wrote:

>> On Jul 15, 2017, at 10:39 PM, Sri h Kolusu  wrote:
>> 
>> Edward, 
>> 
>> Here is a  DFSORT JCL which will give you the desired results. I assumed 
>> that your input file is FB and 80 bytes in length. I also assumed that 
>> your name and domain names are each 30 bytes in length.
>> 
>> //STEP0100 EXEC PGM=SORT 
>> //SYSOUT   DD SYSOUT=* 
>> //SORTIN   DD * 
>> x...@bbb.com  
>> aa...@aaa.com  
>> b...@aa.com  
>> n...@example.com  
>> exampleem...@domain.in  
>> //SORTOUT  DD SYSOUT=* 
>>  INREC PARSE=(%01=(ENDBEFR=C'@',FIXLEN=30),   $ GET FIELD BEFORE @ 
>>   %02=(ENDBEFR=C'.',FIXLEN=30)),  $ GETFIELD BEFORE . 
>>   OVERLAY=(081:%01,   $ PUT NAME AT 81 
>>111:%02)   $ PUT DOMAIN AT 111 
>> 
>>  SORT FIELDS=(081,30,CH,A,$ NAME 
>>   111,30,CH,A),EQUALS $ DOMAIN 
>> 
>>  OUTREC BUILD=(1,80)  $ REMOVE TEMP FIELDS 
>> /*  
>> 
>> The output from this field is
>> 
>> aa...@aaa.com  
>> b...@aa.com  
>> exampleem...@domain.in  
>> n...@example.com  
>> x...@bbb.com  
>> 
>> Further if you have questions please let me know
>> 
>> Thanks,
>> Kolusu
>> DFSORT Development
>> IBM Corporation
>> 
>
>Thanks!… I would never have gotten that without a lot of testing. Also I did 
>not know the first “field” was limited to 30. I learned >several things today.

The part of the email address before the @ is NOT limited to 30 characters. Nor 
is the part after the @. Also, the first part of the address can contain "." 
characters, so the second parse will not work correctly in all cases. 

If you can guarantee that the  part will always be 
present, then it might be safer to parse that part, looking for the text 
between "". But in any case you will 
risk truncating the part before the @ unless you allow for the possibility of 
it being much longer.

-- 
Walt

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[retired mainframer]

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Edward Gould
> Sent: Saturday, July 15, 2017 8:53 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Sort Question
> 
> > On Jul 15, 2017, at 10:39 PM, Sri h Kolusu <skol...@us.ibm.com> wrote:
> >
> > Edward,
> >
> > Here is a  DFSORT JCL which will give you the desired results. I assumed
> > that your input file is FB and 80 bytes in length. I also assumed that
> > your name and domain names are each 30 bytes in length.



> Thanks!… I would never have gotten that without a lot of testing. Also I did 
> not know the
> first “field” was limited to 30. I learned several things today.

The 30 character limit is an assumption.  Use whatever values make sense for 
the data you are processing

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Edward Gould]

> On Jul 15, 2017, at 10:39 PM, Sri h Kolusu  wrote:
> 
> Edward, 
> 
> Here is a  DFSORT JCL which will give you the desired results. I assumed 
> that your input file is FB and 80 bytes in length. I also assumed that 
> your name and domain names are each 30 bytes in length.
> 
> //STEP0100 EXEC PGM=SORT 
> //SYSOUT   DD SYSOUT=* 
> //SORTIN   DD * 
> x...@bbb.com  
> aa...@aaa.com  
> b...@aa.com  
> n...@example.com  
> exampleem...@domain.in  
> //SORTOUT  DD SYSOUT=* 
>  INREC PARSE=(%01=(ENDBEFR=C'@',FIXLEN=30),   $ GET FIELD BEFORE @ 
>   %02=(ENDBEFR=C'.',FIXLEN=30)),  $ GETFIELD BEFORE . 
>   OVERLAY=(081:%01,   $ PUT NAME AT 81 
>111:%02)   $ PUT DOMAIN AT 111 
> 
>  SORT FIELDS=(081,30,CH,A,$ NAME 
>   111,30,CH,A),EQUALS $ DOMAIN 
> 
>  OUTREC BUILD=(1,80)  $ REMOVE TEMP FIELDS 
> /*  
> 
> The output from this field is
> 
> aa...@aaa.com  
> b...@aa.com  
> exampleem...@domain.in  
> n...@example.com  
> x...@bbb.com  
> 
> Further if you have questions please let me know
> 
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
> 

Thanks!… I would never have gotten that without a lot of testing. Also I did 
not know the first “field” was limited to 30. I learned several things today.
Ed


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Sri h Kolusu]

Edward, 

Here is a  DFSORT JCL which will give you the desired results. I assumed 
that your input file is FB and 80 bytes in length. I also assumed that 
your name and domain names are each 30 bytes in length.

//STEP0100 EXEC PGM=SORT 
//SYSOUT   DD SYSOUT=* 
//SORTIN   DD * 
x...@bbb.com 
aa...@aaa.com 
b...@aa.com 
n...@example.com 
exampleem...@domain.in 
//SORTOUT  DD SYSOUT=* 
  INREC PARSE=(%01=(ENDBEFR=C'@',FIXLEN=30),   $ GET FIELD BEFORE @ 
   %02=(ENDBEFR=C'.',FIXLEN=30)),  $ GETFIELD BEFORE . 
   OVERLAY=(081:%01,   $ PUT NAME AT 81 
111:%02)   $ PUT DOMAIN AT 111 
 
  SORT FIELDS=(081,30,CH,A,$ NAME 
   111,30,CH,A),EQUALS $ DOMAIN 
 
  OUTREC BUILD=(1,80)  $ REMOVE TEMP FIELDS 
/*  

The output from this field is

aa...@aaa.com 
b...@aa.com 
exampleem...@domain.in 
n...@example.com 
x...@bbb.com 

Further if you have questions please let me know

Thanks,
Kolusu
DFSORT Development
IBM Corporation



From:   Edward Gould <edgould1...@comcast.net>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   07/15/2017 12:29 PM
Subject:    Sort Question
Sent by:IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU>



One of my users asked me this question and I am not quite clear on how to 
respond.
Given an email address aaa...@bbb.com <mailto:aaa...@bbb.com> (example)

He just wants to sort on the field before the @ and then for a secondary 
sort the bbb field. Both fields are variable in length (not sure what the 
max is).
Any suggestions on how to do this? I have tried RTFM but maybe I am not 
looking at the question correctly .

Ed
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN






--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[retired mainframer]

Look up the PARSE operand of the INREC and OUTREC control statements in the
Application Programming Guide.

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Edward Gould
> Sent: Saturday, July 15, 2017 12:19 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Sort Question
> 
> One of my users asked me this question and I am not quite clear on how to
respond.
> Given an email address aaa...@bbb.com <mailto:aaa...@bbb.com> (example)
> 
> He just wants to sort on the field before the @ and then for a secondary
sort the bbb field.
> Both fields are variable in length (not sure what the max is).
> Any suggestions on how to do this? I have tried RTFM but maybe I am not
looking at the
> question correctly .

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Lizette Koehler]

Does this need to be a mainframe solution?

How many records are involved?

If not a lot of data, I load this type of data to Excel.

The using the DATA tab, I then highlight the column and use the TEXT TO COLUMS
function to break on the @

Do my sorts, then put it back together (using CONCAT function), or leave in
excel for further filtering.

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Edward Gould
> Sent: Saturday, July 15, 2017 12:19 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Sort Question
> 
> One of my users asked me this question and I am not quite clear on how to
> respond.
> Given an email address aaa...@bbb.com <mailto:aaa...@bbb.com> (example)
> 
> He just wants to sort on the field before the @ and then for a secondary sort
> the bbb field. Both fields are variable in length (not sure what the max is).
> Any suggestions on how to do this? I have tried RTFM but maybe I am not
> looking at the question correctly .
> 
> Ed

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort Question

[Scott Barry]

On Sat, 15 Jul 2017 14:18:41 -0500, Edward Gould  
wrote:

>One of my users asked me this question and I am not quite clear on how to 
>respond.
>Given an email address aaa...@bbb.com  (example)
>
>He just wants to sort on the field before the @ and then for a secondary sort 
>the bbb field. Both fields are variable in length (not sure what the max is).
>Any suggestions on how to do this? I have tried RTFM but maybe I am not 
>looking at the question correctly .
>
>Ed

I recall prior threads on parsing IP-address strings, which would be similar in 
context - recommend searching the archives for such reference.

Scott Barry
SBBWorks, Inc. 
http://sbbworks.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Sort Question

[Edward Gould]

One of my users asked me this question and I am not quite clear on how to 
respond.
Given an email address aaa...@bbb.com  (example)

He just wants to sort on the field before the @ and then for a secondary sort 
the bbb field. Both fields are variable in length (not sure what the max is).
Any suggestions on how to do this? I have tried RTFM but maybe I am not looking 
at the question correctly .

Ed
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Martin Packer]

Is this whole limiting number of DD's thing a (lack of) XTIOT thing?

By the way I've thought for some time DFSORT needs a BUCKET ICETOOL 
operator like this. But perhaps as a sample it'd be good. How about it Sri 
Hari?

Cheers, Martin

Martin Packer,
zChampion, Principal Systems Investigator,
Worldwide Banking Center of Excellence, IBM

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker
Blog: 
https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker



From:   Sri h Kolusu skol...@us.ibm.com
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   21/08/2014 00:06
Subject:Re: DF/SORT question (challenge?)
Sent by:IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU



 Try adding close=free to the DD statement

Ed,

When using DFSORT applications, FREE=CLOSE cannot be used on any DD 
statements except DFSPARM. 

Thanks,
Kolusu

IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU wrote on 
08/20/2014 03:24:55 PM:

 From: Ed Gould edgould1...@comcast.net
 To: IBM-MAIN@LISTSERV.UA.EDU
 Date: 08/20/2014 03:28 PM
 Subject: Re: DF/SORT question (challenge?)
 Sent by: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU
 
 On Aug 20, 2014, at 12:54 PM, TonyIcloud-OPERA wrote:
 
  I once allocated a PDSE, and using DF/SORT, wrote 1,000 members to 
  it in one pass of data.  I was tempted to try 10K output members 
  but my beverage wager was already won.  When I get a bit of time, 
  since I already have devised a technique to generate N number of DD 
  statements and OUTFIL statements I'll try 10,000.  I don't recall 
  the maximum number of DD cards??
 
 Try adding close=free to the DD statement
 
 Ed
 
 --
 For IBM-MAIN subscribe / signoff / archive access instructions,
 send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Paul Gilmartin]

On Wed, 20 Aug 2014 19:21:00 -0500, Shane Ginnane wrote:

awk '{print  hlq.$1.file}' dfsort.input

Even OMVS should be able to achieve that. Can't see any reason for a sort.
 
Limit on number of open files?  OP said hundreds or thousands.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[John McKown]

On Thu, Aug 21, 2014 at 7:35 AM, Paul Gilmartin 
000433f07816-dmarc-requ...@listserv.ua.edu wrote:

 On Wed, 20 Aug 2014 19:21:00 -0500, Shane Ginnane wrote:

 awk '{print  hlq.$1.file}' dfsort.input
 
 Even OMVS should be able to achieve that. Can't see any reason for a sort.
 
 Limit on number of open files?  OP said hundreds or thousands.

 -- gil


Good point. Our OMVS MAXFILEPROC is limited to 64000. So hundreds might
not be a problem. But thousands could be since that is a system-wide
limit for all users.


-- 
There is nothing more pleasant than traveling and meeting new people!
Genghis Khan

Maranatha! 
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Shane Ginnane]

On Thu, 21 Aug 2014 07:35:05 -0500, Paul Gilmartin wrote:

Limit on number of open files?  OP said hundreds or thousands.

Rebuke accepted - maybe I can (now) see a need for a sort.

Shane ...

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Shmuel Metz (Seymour J.)]

In op.xkwasag27u3...@hawk01.hsd1.il.comcast.net, on 08/20/2014
   at 09:34 AM, TonyIcloud-OPERA tonybabo...@icloud.com said:

I have a dataset that contains records with a field, by which I need
to create a separate dataset that would contain all occurrences of
that field.

ITYM a separate file for each value in that field. If I were doing
this, I'd sort on the field and have an output exit that allocated and
opened the dataset for each value, closed and freed it on a control
break and told the sort not to create any output.
 
-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 ISO position; see http://patriot.net/~shmuel/resume/brief.html 
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[John McKown]

On Thu, Aug 21, 2014 at 9:37 AM, Shmuel Metz (Seymour J.) 
shmuel+ibm-m...@patriot.net wrote:

 In op.xkwasag27u3...@hawk01.hsd1.il.comcast.net, on 08/20/2014
at 09:34 AM, TonyIcloud-OPERA tonybabo...@icloud.com said:

 I have a dataset that contains records with a field, by which I need
 to create a separate dataset that would contain all occurrences of
 that field.

 ITYM a separate file for each value in that field. If I were doing
 this, I'd sort on the field and have an output exit that allocated and
 opened the dataset for each value, closed and freed it on a control
 break and told the sort not to create any output.


You could even do that in, *gasp*, COBOL with an OUTPUT PROCEDURE IS phrase.



 --
  Shmuel (Seymour J.) Metz, SysProg and JOAT



-- 
There is nothing more pleasant than traveling and meeting new people!
Genghis Khan

Maranatha! 
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[TonyIcloud-OPERA]

 IBM-MAIN@LISTSERV.UA.EDU wrote on
08/20/2014 07:34:48 AM:


From: TonyIcloud-OPERA tonybabo...@icloud.com
To: IBM-MAIN@LISTSERV.UA.EDU
Date: 08/20/2014 07:35 AM
Subject: DF/SORT question (challenge?)
Sent by: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU

I have a dataset that contains records with a field, by which I need to
create a separate dataset that would contain all occurrences of that
field. For example the file, FB100, looks like

value1 other data.
value1 other data.
value1 other data.
value2 other data.
value2 other data.
value2 other data.
value3 other data.
value3 other data.
value3 other data.
value4 other data.
value4 other data.
value4 other data.

My final product must be a series of datasets:

hlq.value1.records
hlq.value2.records
hlq.value3.records
hlq.value4.records

There may be hundreds/thousands of possible values, hence
hundreds/thousands of datasets.  I have accomplished this in 3 phases,
first pass reads the data, uses ICETOOL OCCUR to list the values, second



phase reads the OUTPUT and formats DD statements and OUTFIL OUTREC
statements, the third phase reads the original data to create the

numerous

output files.  I used a newly created PDSE as the output file whereupon
the third phase created several thousand members.

It works, after a fashion, but I'd like a more simple solution.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
Using Opera's mail client: http://www.opera.com/mail/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Hardee, Chuck]

Okay, let me say up front, I know nothing about the abilities of SORT other 
than simple sort this file on these keys and give me the output in this file. 
Okay, we're good?

Here's my question. If (can) sort insert records into its output on a break in 
key?
If so, why not sort the input file and, on each key break, generate an IEBUPDTE 
add control card and the write the record that belong to that card until the 
next key break. Do this for each key break and the resulting output file would 
be an IEUPDTE input stream that would then be fed into an IEBUPDTE job step 
that allocates a new PDS(e) and populates the members.

No need to dynamically create a dataset/member for every key break, just a 
single PS type output file, in key sequence, with control cards as needed.

Admittedly the control cards needed may be a bit more than a simple add/insert 
(don't recall what terminology IEBUPDTE uses, haven't used it in a while), and 
then there's the termination signal for each member, but if the idea has merit, 
well, it's worth a try.

Of course, all bets are off if the OP needs a unique PS type file for each key 
versus a member in a PDS(e) for each key.

Thanks for entertaining this idea.

Chuck

Charles (Chuck) Hardee
Senior Systems Engineer/Database Administration
CCG Information Technology

Thermo Fisher Scientific
300 Industry Drive | Pittsburgh, PA 15275
Phone +1 (724) 517-2633 | Mobile +1 (412) 877-2809 | FAX: +1 (412) 490-9230
chuck.har...@thermofisher.com | www.thermofisher.com

WORLDWIDE CONFIDENTIALITY NOTE: Dissemination, distribution or copying of this 
e-mail or the information herein by anyone other than the intended recipient, 
or an employee or agent of a system responsible for delivering the message to 
the intended recipient, is prohibited. If you are not the intended recipient, 
please inform the sender and delete all copies.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of TonyIcloud-OPERA
Sent: Thursday, August 21, 2014 12:19 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: DF/SORT question (challenge?)

Before I tried the solution cited below (TYVM BTW), I set up a test  
manually with some interesting results.  The input file is not sorted, and  
I'm only doing SORT FIELDS=COPY.  All I want to do is to break up the file  
into a bunch of little files.

Test #1. Read my input file, 229,762 records, lrecl=200/27800 FB , 310 DD  
statements, 310 OUTFIL INCLUDE statements.  RC=0, 310 members created, all  
the data checks out.

Test #2. Read the same input file, 321 DD statements, 321 OUTFIL INCLUDE  
statements, abends:

10.09.26 JOB08830 IEC036I 002-B4,IGC0005E,IDSXSB7A,AA,VS09610,4BB9,SHRE16,  
452
  452 IDSX00S.IDSXSB7.SYSE.FIDX03.PDSE

ICE185A 0 AN S002 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN EXIT  
(PHASE C 3)
Quickref provides:
B4 - Unable to create a system buffer required for PDSE processing.

I retried with regions of 16M, 32M, 0M, same result.  Not sure if our  
local storage police exit chokes off my attempt at 0M.


Interestingly, even though the abend occurred the output PDSE was  
populated with all 321 members however the last 8 members contain 0  
records.  Looks like DF/SORT was tripped up at the 313 mark by some system  
limitation. I'm going to consult with my sysprogs before we go to IBM for  
help.

P.S. My old successful attempt at writing 1,000 members occurred at a  
different company, a much smaller shop, oddly enough.






On Wed, 20 Aug 2014 12:01:00 -0500, Sri h Kolusu skol...@us.ibm.com  
wrote:

 Tony,

 It is quite easy to split the group of records into multiple members.  
 Here
 is a sample JCL which will give you the desired results of splitting the
 first 999 groups of records into a PDSE each containing the group of
 records. I also assumed your Input is already sorted on the field you  
 want
 to split. I assumed that the split field is 44 bytes in length. If it is
 different then you can change it in ALL the places referred by KEYBEGIN.

 This job creates a dynamic JCL which will then be submitted via INTRDR.
 Take a look at the output from Step0200 and then if everything looks ok
 then change the statement
 //SORTOUT  DD SYSOUT=*  to  //SORTOUT  DD SYSOUT=(*,INTRDR),RECFM=FB
 If you have more than 999 groups of records we will copy the rest of the
 records into another file which will then be used as input file to  
 further
 split. I will show you how to build the dynamic JCL's based on that. I
 chose the split to be 999 groups as the maximum number of DD statements
 per job step is 3273, based on the number of single DD statements allowed
 for a TIOT (task input output table) control block size of 64K. This  
 limit
 can be different depending on the installation defined TIOT size. The
 IBM-supplied default TIOT size is 32K.

 //*
 //*  BUILD DYNAMIC OUTFIL CARDS AND DDNAMES FOR EACH GROUP OF RECORDS

Re: DF/SORT question (challenge?)

[TonyIcloud-OPERA]

Chuck, not only will I entertain the idea, I'll try it.
I always have had the option of creating PS datasets but over the years  
I've gotten spoiled for writing to a PDSE because the JCL is simpler.  I  
also became spoiled for getting a quick record count for each member by  
3.5ing the MSL.




On Thu, 21 Aug 2014 11:28:51 -0500, Hardee, Chuck  
chuck.har...@thermofisher.com wrote:


Okay, let me say up front, I know nothing about the abilities of SORT  
other than simple sort this file on these keys and give me the output in  
this file. Okay, we're good?


Here's my question. If (can) sort insert records into its output on a  
break in key?
If so, why not sort the input file and, on each key break, generate an  
IEBUPDTE add control card and the write the record that belong to that  
card until the next key break. Do this for each key break and the  
resulting output file would be an IEUPDTE input stream that would then  
be fed into an IEBUPDTE job step that allocates a new PDS(e) and  
populates the members.


No need to dynamically create a dataset/member for every key break, just  
a single PS type output file, in key sequence, with control cards as  
needed.


Admittedly the control cards needed may be a bit more than a simple  
add/insert (don't recall what terminology IEBUPDTE uses, haven't used it  
in a while), and then there's the termination signal for each member,  
but if the idea has merit, well, it's worth a try.


Of course, all bets are off if the OP needs a unique PS type file for  
each key versus a member in a PDS(e) for each key.


Thanks for entertaining this idea.

Chuck

Charles (Chuck) Hardee
Senior Systems Engineer/Database Administration
CCG Information Technology

Thermo Fisher Scientific
300 Industry Drive | Pittsburgh, PA 15275
Phone +1 (724) 517-2633 | Mobile +1 (412) 877-2809 | FAX: +1 (412)  
490-9230

chuck.har...@thermofisher.com | www.thermofisher.com

WORLDWIDE CONFIDENTIALITY NOTE: Dissemination, distribution or copying  
of this e-mail or the information herein by anyone other than the  
intended recipient, or an employee or agent of a system responsible for  
delivering the message to the intended recipient, is prohibited. If you  
are not the intended recipient, please inform the sender and delete all  
copies.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On  
Behalf Of TonyIcloud-OPERA

Sent: Thursday, August 21, 2014 12:19 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: DF/SORT question (challenge?)

Before I tried the solution cited below (TYVM BTW), I set up a test
manually with some interesting results.  The input file is not sorted,  
and
I'm only doing SORT FIELDS=COPY.  All I want to do is to break up the  
file

into a bunch of little files.

Test #1. Read my input file, 229,762 records, lrecl=200/27800 FB , 310 DD
statements, 310 OUTFIL INCLUDE statements.  RC=0, 310 members created,  
all

the data checks out.

Test #2. Read the same input file, 321 DD statements, 321 OUTFIL INCLUDE
statements, abends:

10.09.26 JOB08830 IEC036I  
002-B4,IGC0005E,IDSXSB7A,AA,VS09610,4BB9,SHRE16,

452
  452 IDSX00S.IDSXSB7.SYSE.FIDX03.PDSE

ICE185A 0 AN S002 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN EXIT
(PHASE C 3)
Quickref provides:
B4 - Unable to create a system buffer required for PDSE processing.

I retried with regions of 16M, 32M, 0M, same result.  Not sure if our
local storage police exit chokes off my attempt at 0M.


Interestingly, even though the abend occurred the output PDSE was
populated with all 321 members however the last 8 members contain 0
records.  Looks like DF/SORT was tripped up at the 313 mark by some  
system

limitation. I'm going to consult with my sysprogs before we go to IBM for
help.

P.S. My old successful attempt at writing 1,000 members occurred at a
different company, a much smaller shop, oddly enough.






On Wed, 20 Aug 2014 12:01:00 -0500, Sri h Kolusu skol...@us.ibm.com
wrote:


Tony,

It is quite easy to split the group of records into multiple members.
Here
is a sample JCL which will give you the desired results of splitting the
first 999 groups of records into a PDSE each containing the group of
records. I also assumed your Input is already sorted on the field you
want
to split. I assumed that the split field is 44 bytes in length. If it is
different then you can change it in ALL the places referred by KEYBEGIN.

This job creates a dynamic JCL which will then be submitted via INTRDR.
Take a look at the output from Step0200 and then if everything looks ok
then change the statement
//SORTOUT  DD SYSOUT=*  to  //SORTOUT  DD SYSOUT=(*,INTRDR),RECFM=FB
If you have more than 999 groups of records we will copy the rest of the
records into another file which will then be used as input file to
further
split. I will show you how to build the dynamic JCL's based on that. I
chose the split to be 999 groups as the maximum number of DD statements
per job step is 3273, based

Re: DF/SORT question (challenge?)

[TonyIcloud-OPERA]

Geez, I really should read:
1. the sample provided.
2. the book

RC=0

Now I need to find out more about ODMAXBF.  I feel like the guy who bought  
the toaster and didn't plug it in.







On Thu, 21 Aug 2014 11:35:25 -0500, Sri h Kolusu skol...@us.ibm.com  
wrote:



Tony,

Did you have the override of ODMAXBF?  In my sample JCL I have coded it  
as

ODMAXBF=100K overriding the default value of 2M. Can you rerun the job
with the ODMAXBF parm?

Thanks,
Kolusu
DFSORT Development
IBM Corporation

IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU wrote on
08/21/2014 09:18:40 AM:


From: TonyIcloud-OPERA tonybabo...@icloud.com
To: IBM-MAIN@LISTSERV.UA.EDU
Date: 08/21/2014 09:25 AM
Subject: Re: DF/SORT question (challenge?)
Sent by: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU

Before I tried the solution cited below (TYVM BTW), I set up a test
manually with some interesting results.  The input file is not sorted,

and

I'm only doing SORT FIELDS=COPY.  All I want to do is to break up the

file

into a bunch of little files.

Test #1. Read my input file, 229,762 records, lrecl=200/27800 FB , 310

DD

statements, 310 OUTFIL INCLUDE statements.  RC=0, 310 members created,

all

the data checks out.

Test #2. Read the same input file, 321 DD statements, 321 OUTFIL INCLUDE



statements, abends:

10.09.26 JOB08830 IEC036I

002-B4,IGC0005E,IDSXSB7A,AA,VS09610,4BB9,SHRE16,

452
  452 IDSX00S.IDSXSB7.SYSE.FIDX03.PDSE

ICE185A 0 AN S002 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN EXIT



(PHASE C 3)
Quickref provides:
B4 - Unable to create a system buffer required for PDSE processing.

I retried with regions of 16M, 32M, 0M, same result.  Not sure if our
local storage police exit chokes off my attempt at 0M.


Interestingly, even though the abend occurred the output PDSE was
populated with all 321 members however the last 8 members contain 0
records.  Looks like DF/SORT was tripped up at the 313 mark by some

system

limitation. I'm going to consult with my sysprogs before we go to IBM

for

help.

P.S. My old successful attempt at writing 1,000 members occurred at a
different company, a much smaller shop, oddly enough.






On Wed, 20 Aug 2014 12:01:00 -0500, Sri h Kolusu skol...@us.ibm.com
wrote:

 Tony,

 It is quite easy to split the group of records into multiple members.
 Here
 is a sample JCL which will give you the desired results of splitting

the

 first 999 groups of records into a PDSE each containing the group of
 records. I also assumed your Input is already sorted on the field you
 want
 to split. I assumed that the split field is 44 bytes in length. If it

is

 different then you can change it in ALL the places referred by

KEYBEGIN.


 This job creates a dynamic JCL which will then be submitted via

INTRDR.

 Take a look at the output from Step0200 and then if everything looks

ok

 then change the statement
 //SORTOUT  DD SYSOUT=*  to  //SORTOUT  DD SYSOUT=(*,INTRDR),RECFM=FB
 If you have more than 999 groups of records we will copy the rest of

the

 records into another file which will then be used as input file to
 further
 split. I will show you how to build the dynamic JCL's based on that. I
 chose the split to be 999 groups as the maximum number of DD

statements

 per job step is 3273, based on the number of single DD statements

allowed

 for a TIOT (task input output table) control block size of 64K. This
 limit
 can be different depending on the installation defined TIOT size. The
 IBM-supplied default TIOT size is 32K.



//*

 //*  BUILD DYNAMIC OUTFIL CARDS AND DDNAMES FOR EACH GROUP OF RECORDS

*



//*

 //STEP0100 EXEC PGM=SORT
 //SYSOUT   DD SYSOUT=*
 //SORTIN   DD DISP=SHR,DSN=Your Input FB 100 Byte file
 //*
 //OFCARDS  DD DSN=C,DISP=(,PASS),SPACE=(CYL,(20,20),RLSE)
 //DDNAMES  DD DSN=D,DISP=(,PASS),SPACE=(CYL,(20,20),RLSE)
 //SORTOUT  DD DUMMY
 //SYSINDD *
   OPTION COPY,NULLOUT=RC4
   OUTREC IFTHEN=(WHEN=GROUP,KEYBEGIN=(1,44),PUSH=(101:ID=4))
  OUTFIL FNAMES=OFCARDS,REMOVECC,NODETAIL,BUILD=(80X),
   INCLUDE=(101,4,ZD,LT,1000),
   SECTIONS=(101,4,
   TRAILER3=(3:'OUTFIL FNAMES=OUTF',101,4,',BUILD=(1,100),',
C'INCLUDE=(101,4,ZD,EQ,',101,4,')')),
   TRAILER1=(3:'OUTFIL FNAMES=NGRP',101,4,',SAVE')
  OUTFIL FNAMES=DDNAMES,REMOVECC,NODETAIL,BUILD=(80X),
   INCLUDE=(101,4,ZD,LT,1000),
   SECTIONS=(101,4,
   TRAILER3=('//OUTF',101,4,' DD ',
 'DISP=SHR,DSN=Your.Split.PDSE(OUTF',101,4,')')),
   TRAILER1=('//NGRP',101,4,' DD ',
 'DSN=HLQ.TONYCLD.NGRP',101,4,','/,
 '//',15:'DISP=(NEW,CATLG,DELETE),',/,
 '//',15:'SPACE=(CYL,(100,40),RLSE)',/,
 '//*')
 //*


//*

 //*  SUBMIT THE SPLIT JOB TO INTRDR WITH THE ABOVE OUTPUT

Re: DF/SORT question (challenge?)

[Sri h Kolusu]

Tony,

ODMAXBF specifies the maximum buffer space DFSORT can use for each OUTFIL 
data set. Since you are creating so many OUTFIL files, it is a good idea 
to lower the buffer space for each OUTFIL.  Check this link which explains 
about ODMAXBF parameter in detail (search for ODMAXBF on that page)

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ice1ca60/3.14

Thanks
Kolusu
DFSORT Development
IBM Corporation



From:   TonyIcloud-OPERA tonybabo...@icloud.com
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   08/21/2014 09:43 AM
Subject:Re: DF/SORT question (challenge?)
Sent by:IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU



Geez, I really should read:
1. the sample provided.
2. the book

RC=0

Now I need to find out more about ODMAXBF.  I feel like the guy who bought 
 
the toaster and didn't plug it in.






On Thu, 21 Aug 2014 11:35:25 -0500, Sri h Kolusu skol...@us.ibm.com 
wrote:

 Tony,

 Did you have the override of ODMAXBF?  In my sample JCL I have coded it 
 as
 ODMAXBF=100K overriding the default value of 2M. Can you rerun the job
 with the ODMAXBF parm?

 Thanks,
 Kolusu
 DFSORT Development
 IBM Corporation

 IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU wrote on
 08/21/2014 09:18:40 AM:

 From: TonyIcloud-OPERA tonybabo...@icloud.com
 To: IBM-MAIN@LISTSERV.UA.EDU
 Date: 08/21/2014 09:25 AM
 Subject: Re: DF/SORT question (challenge?)
 Sent by: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU

 Before I tried the solution cited below (TYVM BTW), I set up a test
 manually with some interesting results.  The input file is not sorted,
 and
 I'm only doing SORT FIELDS=COPY.  All I want to do is to break up the
 file
 into a bunch of little files.

 Test #1. Read my input file, 229,762 records, lrecl=200/27800 FB , 310
 DD
 statements, 310 OUTFIL INCLUDE statements.  RC=0, 310 members created,
 all
 the data checks out.

 Test #2. Read the same input file, 321 DD statements, 321 OUTFIL 
INCLUDE

 statements, abends:

 10.09.26 JOB08830 IEC036I
 002-B4,IGC0005E,IDSXSB7A,AA,VS09610,4BB9,SHRE16,
 452
   452 IDSX00S.IDSXSB7.SYSE.FIDX03.PDSE

 ICE185A 0 AN S002 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN 
EXIT

 (PHASE C 3)
 Quickref provides:
 B4 - Unable to create a system buffer required for PDSE processing.

 I retried with regions of 16M, 32M, 0M, same result.  Not sure if our
 local storage police exit chokes off my attempt at 0M.


 Interestingly, even though the abend occurred the output PDSE was
 populated with all 321 members however the last 8 members contain 0
 records.  Looks like DF/SORT was tripped up at the 313 mark by some
 system
 limitation. I'm going to consult with my sysprogs before we go to IBM
 for
 help.

 P.S. My old successful attempt at writing 1,000 members occurred at a
 different company, a much smaller shop, oddly enough.






 On Wed, 20 Aug 2014 12:01:00 -0500, Sri h Kolusu skol...@us.ibm.com
 wrote:

  Tony,
 
  It is quite easy to split the group of records into multiple members.
  Here
  is a sample JCL which will give you the desired results of splitting
 the
  first 999 groups of records into a PDSE each containing the group of
  records. I also assumed your Input is already sorted on the field you
  want
  to split. I assumed that the split field is 44 bytes in length. If it
 is
  different then you can change it in ALL the places referred by
 KEYBEGIN.
 
  This job creates a dynamic JCL which will then be submitted via
 INTRDR.
  Take a look at the output from Step0200 and then if everything looks
 ok
  then change the statement
  //SORTOUT  DD SYSOUT=*  to  //SORTOUT  DD SYSOUT=(*,INTRDR),RECFM=FB
  If you have more than 999 groups of records we will copy the rest of
 the
  records into another file which will then be used as input file to
  further
  split. I will show you how to build the dynamic JCL's based on that. 
I
  chose the split to be 999 groups as the maximum number of DD
 statements
  per job step is 3273, based on the number of single DD statements
 allowed
  for a TIOT (task input output table) control block size of 64K. This
  limit
  can be different depending on the installation defined TIOT size. The
  IBM-supplied default TIOT size is 32K.
 
 
 //*
  //*  BUILD DYNAMIC OUTFIL CARDS AND DDNAMES FOR EACH GROUP OF RECORDS
 *
 
 //*
  //STEP0100 EXEC PGM=SORT
  //SYSOUT   DD SYSOUT=*
  //SORTIN   DD DISP=SHR,DSN=Your Input FB 100 Byte file
  //*
  //OFCARDS  DD DSN=C,DISP=(,PASS),SPACE=(CYL,(20,20),RLSE)
  //DDNAMES  DD DSN=D,DISP=(,PASS),SPACE=(CYL,(20,20),RLSE)
  //SORTOUT  DD DUMMY
  //SYSINDD *
OPTION COPY,NULLOUT=RC4
OUTREC IFTHEN=(WHEN=GROUP,KEYBEGIN=(1,44),PUSH=(101:ID=4))
   OUTFIL FNAMES=OFCARDS,REMOVECC,NODETAIL,BUILD=(80X),
INCLUDE=(101,4,ZD,LT,1000),
SECTIONS=(101,4,
TRAILER3=(3:'OUTFIL FNAMES=OUTF',101,4,',BUILD=(1,100

Re: DF/SORT question (challenge?)

[Sri h Kolusu]

Here's my question. If (can) sort insert records into its output on a 
break in key?
If so, why not sort the input file and, on each key break, generate an 
IEBUPDTE add control card and the write the record that belong to that 
card until the next key break. Do this for each key break and the 
resulting output file would be an IEUPDTE input stream that would then be 
fed into an IEBUPDTE job step that allocates a new PDS(e) and populates 
the members.

Chuck,

DFSORT is quite capable of generating the IEBUPDTE cards on key break. My 
sample JCL is doing a similar thing but generating the DFSORT OUTFIL 
control cards based on the key break.  If you want I can show you how to 
generate the IEBUPDTE sysin cards or for that matter any IBM utility 
control cards using DFSORT.  Send me an email and we can discuss it 
offline.

Thanks,
Kolusu
DFSORT Development

IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU wrote on 
08/21/2014 09:28:51 AM:

 From: Hardee, Chuck chuck.har...@thermofisher.com
 To: IBM-MAIN@LISTSERV.UA.EDU
 Date: 08/21/2014 09:29 AM
 Subject: Re: DF/SORT question (challenge?)
 Sent by: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU
 
 Okay, let me say up front, I know nothing about the abilities of 
 SORT other than simple sort this file on these keys and give me the 
 output in this file. Okay, we're good?
 
 Here's my question. If (can) sort insert records into its output on 
 a break in key?
 If so, why not sort the input file and, on each key break, generate 
 an IEBUPDTE add control card and the write the record that belong to
 that card until the next key break. Do this for each key break and 
 the resulting output file would be an IEUPDTE input stream that 
 would then be fed into an IEBUPDTE job step that allocates a new PDS
 (e) and populates the members.
 
 No need to dynamically create a dataset/member for every key break, 
 just a single PS type output file, in key sequence, with control 
 cards as needed.
 
 Admittedly the control cards needed may be a bit more than a simple 
 add/insert (don't recall what terminology IEBUPDTE uses, haven't 
 used it in a while), and then there's the termination signal for 
 each member, but if the idea has merit, well, it's worth a try.
 
 Of course, all bets are off if the OP needs a unique PS type file 
 for each key versus a member in a PDS(e) for each key.
 
 Thanks for entertaining this idea.
 
 Chuck
 
 Charles (Chuck) Hardee
 Senior Systems Engineer/Database Administration
 CCG Information Technology
 
 Thermo Fisher Scientific
 300 Industry Drive | Pittsburgh, PA 15275
 Phone +1 (724) 517-2633 | Mobile +1 (412) 877-2809 | FAX: +1 (412) 
490-9230
 chuck.har...@thermofisher.com | www.thermofisher.com
 
 WORLDWIDE CONFIDENTIALITY NOTE: Dissemination, distribution or 
 copying of this e-mail or the information herein by anyone other 
 than the intended recipient, or an employee or agent of a system 
 responsible for delivering the message to the intended recipient, is
 prohibited. If you are not the intended recipient, please inform the
 sender and delete all copies.
 
 -Original Message-
 From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU
 ] On Behalf Of TonyIcloud-OPERA
 Sent: Thursday, August 21, 2014 12:19 PM
 To: IBM-MAIN@LISTSERV.UA.EDU
 Subject: Re: DF/SORT question (challenge?)
 
 Before I tried the solution cited below (TYVM BTW), I set up a test 
 manually with some interesting results.  The input file is not sorted, 
and 
 I'm only doing SORT FIELDS=COPY.  All I want to do is to break up the 
file 
 into a bunch of little files.
 
 Test #1. Read my input file, 229,762 records, lrecl=200/27800 FB , 310 
DD 
 statements, 310 OUTFIL INCLUDE statements.  RC=0, 310 members created, 
all 
 the data checks out.
 
 Test #2. Read the same input file, 321 DD statements, 321 OUTFIL INCLUDE 
 
 statements, abends:
 
 10.09.26 JOB08830 IEC036I 
002-B4,IGC0005E,IDSXSB7A,AA,VS09610,4BB9,SHRE16, 
 452
   452 IDSX00S.IDSXSB7.SYSE.FIDX03.PDSE
 
 ICE185A 0 AN S002 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN EXIT 
 
 (PHASE C 3)
 Quickref provides:
 B4 - Unable to create a system buffer required for PDSE processing.
 
 I retried with regions of 16M, 32M, 0M, same result.  Not sure if our 
 local storage police exit chokes off my attempt at 0M.
 
 
 Interestingly, even though the abend occurred the output PDSE was 
 populated with all 321 members however the last 8 members contain 0 
 records.  Looks like DF/SORT was tripped up at the 313 mark by some 
system 
 limitation. I'm going to consult with my sysprogs before we go to IBM 
for 
 help.
 
 P.S. My old successful attempt at writing 1,000 members occurred at a 
 different company, a much smaller shop, oddly enough.
 
 
 
 
 
 
 On Wed, 20 Aug 2014 12:01:00 -0500, Sri h Kolusu skol...@us.ibm.com 
 wrote:
 
  Tony,
 
  It is quite easy to split the group of records into multiple members. 
  Here
  is a sample JCL which will give you the desired results

Re: DF/SORT question (challenge?)

[Shmuel Metz (Seymour J.)]

In 0944307717662460.wa.paulgboulderaim@listserv.ua.edu, on
08/20/2014
   at 12:13 PM, Paul Gilmartin
000433f07816-dmarc-requ...@listserv.ua.edu said:

Strange DSN syntax,

Have you checked the TRAILER3 keyword of OUTFIL in the DF/SORT
documentation?
 
-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 ISO position; see http://patriot.net/~shmuel/resume/brief.html 
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

DF/SORT question (challenge?)

[TonyIcloud-OPERA]

I have a dataset that contains records with a field, by which I need to  
create a separate dataset that would contain all occurrences of that  
field. For example the file, FB100, looks like


value1 other data.
value1 other data.
value1 other data.
value2 other data.
value2 other data.
value2 other data.
value3 other data.
value3 other data.
value3 other data.
value4 other data.
value4 other data.
value4 other data.

My final product must be a series of datasets:

hlq.value1.records
hlq.value2.records
hlq.value3.records
hlq.value4.records

There may be hundreds/thousands of possible values, hence  
hundreds/thousands of datasets.  I have accomplished this in 3 phases,  
first pass reads the data, uses ICETOOL OCCUR to list the values, second  
phase reads the OUTPUT and formats DD statements and OUTFIL OUTREC  
statements, the third phase reads the original data to create the numerous  
output files.  I used a newly created PDSE as the output file whereupon  
the third phase created several thousand members.


It works, after a fashion, but I'd like a more simple solution.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: DF/SORT question (challenge?)

[Sam Siegel]

Have you considered writing a program which dynamically allocates the
output files?


On Wed, Aug 20, 2014 at 7:34 AM, TonyIcloud-OPERA tonybabo...@icloud.com
wrote:

 I have a dataset that contains records with a field, by which I need to
 create a separate dataset that would contain all occurrences of that field.
 For example the file, FB100, looks like

 value1 other data.
 value1 other data.
 value1 other data.
 value2 other data.
 value2 other data.
 value2 other data.
 value3 other data.
 value3 other data.
 value3 other data.
 value4 other data.
 value4 other data.
 value4 other data.

 My final product must be a series of datasets:

 hlq.value1.records
 hlq.value2.records
 hlq.value3.records
 hlq.value4.records

 There may be hundreds/thousands of possible values, hence
 hundreds/thousands of datasets.  I have accomplished this in 3 phases,
 first pass reads the data, uses ICETOOL OCCUR to list the values, second
 phase reads the OUTPUT and formats DD statements and OUTFIL OUTREC
 statements, the third phase reads the original data to create the numerous
 output files.  I used a newly created PDSE as the output file whereupon the
 third phase created several thousand members.

 It works, after a fashion, but I'd like a more simple solution.


 --
 For IBM-MAIN subscribe / signoff / archive access instructions,
 send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN