Re: What am I doing wrong with BPXWUNIX sort?

[Charles Mills]

Yes, that is exactly my impression. I have not run exhaustive experiments but I 
think that is it. Yes, for reasons related to other logic, this program 
contains Index. = "" but Sri's and my previous successful use of BPXWUNIX sort 
do not.

CM

On Thu, 7 Mar 2024 17:17:23 +, Jeremy Nicoll 
 wrote:

>On Thu, 7 Mar 2024, at 16:54, Charles Mills wrote:
>> Thank you! THAT is the clue I needed. I need to quote the stem names.
>> Passing plain Index. passes "", the value of Index., to sort.
>
>
>So... the difference between your code & Sri's is that you'd
>initialised index. = ""   whereas his didn't have a default
>value for the stems? ...
>
>... because his example didn't have quotes around stem names?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Jeremy Nicoll]

On Thu, 7 Mar 2024, at 16:54, Charles Mills wrote:
> Thank you! THAT is the clue I needed. I need to quote the stem names. 
> Passing plain Index. passes "", the value of Index., to sort.


So... the difference between your code & Sri's is that you'd 
initialised index. = ""   whereas his didn't have a default 
value for the stems? ...

... because his example didn't have quotes around stem names?

-- 
Jeremy Nicoll - my opinions are my own.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Charles Mills]

Thank you! THAT is the clue I needed. I need to quote the stem names. Passing 
plain Index. passes "", the value of Index., to sort.

Problem solved. Thanks all.

CM

On Thu, 7 Mar 2024 16:22:02 +, Alan Young  wrote:

>I think I have always used the position specification format. In a couple of 
>processes I ran before, I have
>
>xrc = BPXWUNIX("/bin/sort -bdu -k1.1,1.11 ","pl.","pls.")

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Alan Young]

I think I have always used the position specification format. In a couple of 
processes I ran before, I have

xrc = BPXWUNIX("/bin/sort -bdu -k1.1,1.11 ","pl.","pls.")

xrc = BPXWUNIX("/bin/sort -bd -k1.10,1.26 -k2.1,2.8","d.","ds.")

Maybe sort is aborting with a return code (RC, retval, etc.) and no message? 

Alan

-Original Message-
From: IBM Mainframe Discussion List 
Sent: Mar 7, 2024 9:04 AM
To: 
Subject: Re: What am I doing wrong with BPXWUNIX sort?

Thanks all. The mystery deepens.

Using the same stem variable should not be the problem. I have done that before 
successfully, and the sort command documentation talks about how it uses a 
temporary file to avoid clobbering the input data if the files are the same.

BUT ... changing to a different stem variable revealed that sort is not 
populating my output file at all. The new stem variable was uninitialized after 
the sort! THAT is why Index. is unchanged -- sort is not writing to my output 
file/stem at all!

So that leaves the question: why is sort not populating my output stem? I know 
sort is available and actually running because in one test I fat-fingered the 
-k2 and got three messages in stderr.

Why would sort not populate the output stem?

Charles


On Thu, 7 Mar 2024 10:15:10 +0800, David Crayford wrote:

>Youre using the same stem variable for input and output. Use a specific 
>stdout. stem and see if that fixes it. Kolusus snippet works for me.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Charles Mills]

Thanks all. The mystery deepens.

Using the same stem variable should not be the problem. I have done that before 
successfully, and the sort command documentation talks about how it uses a 
temporary file to avoid clobbering the input data if the files are the same.

BUT ... changing to a different stem variable revealed that sort is not 
populating my output file at all. The new stem variable was uninitialized after 
the sort! THAT is why Index. is unchanged -- sort is not writing to my output 
file/stem at all! 

So that leaves the question: why is sort not populating my output stem? I know 
sort is available and actually running because in one test I fat-fingered the 
-k2 and got three messages in stderr.

Why would sort not populate the output stem?

Charles

 
On Thu, 7 Mar 2024 10:15:10 +0800, David Crayford  wrote:

>You’re using the same stem variable for input and output. Use a specific 
>stdout. stem and see if that fixes it. Kolusu’s snippet works for me. 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Schmitt, Michael]

Not the question you asked, but...


sort_stem = 'index.'
call sort 6, ???(see comments after code)


/* Sort sort_stem using combsort */
sort: procedure,
   expose (sort_stem)
   arg key_start, key_length
   size = value(sort_stem'0')
   gap  = size
   do until switches = 0 & gap = 1
  gap = gap % 1.3
  select
 when gap =  0 then gap =  1
 when gap =  9 then gap = 11
 when gap = 10 then gap = 11
 otherwise
 end
  switches = 0
  do i = 1 to (size - gap)
 j = i + gap
 if substr(value(sort_stem'i'), key_start, key_length) >,
substr(value(sort_stem'j'), key_start, key_length) then do
/* swap the entries */
hold = value(sort_stem'i')   /* hold   = stem.i */
x = value(sort_stem'i', value(sort_stem'j')) /* stem.i = stem.j */
x = value(sort_stem'j', hold)/* stem.j = hold   */
switches = switches + 1
end
 end
  end
   return


The problem is that as written, the code expects the key_length to be the same 
across all elements of the stem. If yours are variable length then the code 
will need to be adjusted. Maybe in your case it would need to be comparing 
subwords instead of substr.

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Charles Mills
Sent: Wednesday, March 6, 2024 6:10 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: What am I doing wrong with BPXWUNIX sort?

I am trying to sort a Rexx "array" starting with the second "word" of the 
variables. My "array" is in Index.n and contains records of the form  where  is 0001, 0002, 0003, etc. and string is 2 to 5 
Rexx "words."

Here's my Rexx code:

  Say "Before sort" Index.0 Index.1 Index.2 Index.3
  stdout.0 = 0
  stderr.0 = 0
  Call BPXWUNIX "/bin/sort -k2",Index.,Index.,stderr.
  Do i = 1 to stderr.0
Say "Sort error:" stderr.i
End i
  Say "After  sort" Index.0 Index.1 Index.2 Index.3

And here is the output:

Before sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation
After  sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation

My expectation is that -k2 would have caused sort to sort on the "descriptive 
strings" (ignoring the ) but obviously that is not what has happened. What 
am I doing wrong? (I have tried both -k2 and -k 2, and also -k1 and +1 -- all 
with the same results.)

Thanks,
Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[David Crayford]

You’re using the same stem variable for input and output. Use a specific 
stdout. stem and see if that fixes it. Kolusu’s snippet works for me. 

> On 7 Mar 2024, at 8:41 am, Charles Mills  wrote:
> 
> Thanks. As I said, I have tried both -k2 and -k 2, and also -k1 and +1, all 
> with the same result.
> 
> CM
> 
> On Thu, 7 Mar 2024 00:27:21 +, Sri Hari Kolusu  wrote:
> 
>> Charles,
>> 
>> Try a space after k.
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Dale R. Smith]

On Wed, 6 Mar 2024 18:10:28 -0600, Charles Mills  wrote:

>I am trying to sort a Rexx "array" starting with the second "word" of the 
>variables. My "array" is in Index.n and contains records of the form some descriptive string> where  is 0001, 0002, 0003, etc. and string is 2 
>to 5 Rexx "words."
>
>Here's my Rexx code:
>
>  Say "Before sort" Index.0 Index.1 Index.2 Index.3
>  stdout.0 = 0
>  stderr.0 = 0
>  Call BPXWUNIX "/bin/sort -k2",Index.,Index.,stderr.
>  Do i = 1 to stderr.0
>Say "Sort error:" stderr.i
>End i
>  Say "After  sort" Index.0 Index.1 Index.2 Index.3
>
>And here is the output:
>
>Before sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation 
>After  sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation 
>
>My expectation is that -k2 would have caused sort to sort on the "descriptive 
>strings" (ignoring the ) but obviously that is not what has happened. What 
>am I doing wrong? (I have tried both -k2 and -k 2, and also -k1 and +1 -- all 
>with the same results.)
>
>Thanks,
>Charles

Use a different Stem Name for the output, it can't be the same name as the 
input.

-- 
Dale R. Smith

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Charles Mills]

Thanks. As I said, I have tried both -k2 and -k 2, and also -k1 and +1, all 
with the same result.

CM

On Thu, 7 Mar 2024 00:27:21 +, Sri Hari Kolusu  wrote:

>Charles,
>
>Try a space after k.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Sri Hari Kolusu]

Charles,

Your example data

/* REXX */
stdin.0=3
stdin.1="0001 Main Check"
stdin.2="0002 OMVS (FTP Session)"
stdin.3="0003 C Validation"

cmd="sort -k 2"
call bpxwunix cmd,stdin.,stdout.,stderr.

say "stdout:"
say "==="
do i=1 to stdout.0
  say stdout.i
end

say "stderr:"
say "==="
do i=1 to stderr.0
   say stderr.i
end

produces


stdout:
===
0003 C Validation
0001 Main Check
0002 OMVS (FTP Session)
stderr:
===

Thanks,
Kolusu


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: What am I doing wrong with BPXWUNIX sort?

[Sri Hari Kolusu]

Charles,

Try a space after k.

Something like this

/* REXX */
stdin.0=5
stdin.1="KIJJ 3"
stdin.2="KQWR 1"
stdin.3="ADGF 2"
stdin.4="OEPE 6"
stdin.5="VNVV 5"

cmd="sort -k 2"
call bpxwunix cmd,stdin.,stdout.,stderr.

say "stdout:"
say "==="
do i=1 to stdout.0
  say stdout.i
end

say "stderr:"
say "==="
do i=1 to stderr.0
   say stderr.i
end

Thanks,
Kolusu


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

What am I doing wrong with BPXWUNIX sort?

[Charles Mills]

I am trying to sort a Rexx "array" starting with the second "word" of the 
variables. My "array" is in Index.n and contains records of the form  where  is 0001, 0002, 0003, etc. and string is 2 to 5 
Rexx "words."

Here's my Rexx code:

  Say "Before sort" Index.0 Index.1 Index.2 Index.3
  stdout.0 = 0
  stderr.0 = 0
  Call BPXWUNIX "/bin/sort -k2",Index.,Index.,stderr.
  Do i = 1 to stderr.0
Say "Sort error:" stderr.i
End i
  Say "After  sort" Index.0 Index.1 Index.2 Index.3

And here is the output:

Before sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation 
After  sort 8 0001 Main Check 0002 OMVS (FTP Session) 0003 C Validation 

My expectation is that -k2 would have caused sort to sort on the "descriptive 
strings" (ignoring the ) but obviously that is not what has happened. What 
am I doing wrong? (I have tried both -k2 and -k 2, and also -k1 and +1 -- all 
with the same results.)

Thanks,
Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Generating output in SORT with a time value

[Martin Packer]

An E15 exit could do it – except it might be called too frequently. But 
probably STCKE would work. I think that’d be expensive, though.

Cheers, Martin

From: IBM Mainframe Discussion List  on behalf of 
Massimo Biancucci 
Date: Thursday, 1 February 2024 at 15:12
To: IBM-MAIN@LISTSERV.UA.EDU 
Subject: [EXTERNAL] Re: Generating output in SORT with a time value
Billy,

as Sri said there's no way to ask SORT tools to put a different TS on every
record.
Hereby a "try" to start with an arbitrary TS (in TOD format to generate as
you like) and then generate a number of consecutive TS part (HHMMSSuu)
with a distance of 1 us.

It's a two steps and probably can be shrinked into one.
Consider it as an idea to further develop.
This doesn't guarantee any uniqueness between two different runs or single
run generating more than "1 day" number of records.
Hope this helps.
Max

//*---*
//* LET'S GENERATE A ARBITRARY NUMBERS OF ROWS WITH THE SEQNUM IN *
//* BINARY FORMAT AND A FIX TOD. LET'S CALL IT BASE TOD.  *
//*---*
//ST001EXEC  PGM=SORT
//SYSOUTDD   SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD DSN=&,DISP=(,PASS),SPACE=(TRK,(10,10),RLSE)
//SYSIN DD   *
 SORT FIELDS=COPY
 OUTFIL REPEAT=10,
  BUILD=(SEQNUM,4,BI,START=0,X'DE956F5F25C18000')
*2024-01-31-23.59.59.999000
* BUILD=(SEQNUM,4,BI,X'DE9542AB0E80')
*2024-01-31-20.40.00.00
/*
//*---*
//* TOD IS A 8 BYTES BINARY WHERE BIT 51 IS 1 US  *
//* IT MEANS YOU HAVE TO DIVIDE TOD BY 4096 TO CALC NUM OF US *
//* SO:   *
//*---*
//* +1291714560   1 YEAR  *
//* +3538944  1 DAY   *
//* +147456   1 HOUR  *
//* +24576000 1 MINUTE*
//* +409600   1 SECOND*
//* +4096 1 US*
//*---*
//* SORT CAN TRANSFORM TOD TO DATE OR TIME.   *
//* FOR TIME THE MAXIMUM PRECISION IS HUNDREDTHS SECOND.  *
//* WE NEED TO CALCULATE THE LAST 4 DIGITS OF MICROSECONDS.   *
//*---*
//* FIRST ADD SEQNUM US TO THE BASE TOD.  *
//* EVERY RECORD WILL HAVE ONE US MORE.   *
//* LET'S PRINT TOD AS TC4: HHMMSSHH. *
//* LET'S CALC LAST 4 DIGITS AS REMAINDER OF TOD BY 4096. *
//* THIS IS THE VALUE IN TOD US UNITS. DIVIDE IT BY 4096 TO GET US.   *
//*---*
//* WE USE THIS MECHANISM TO USE TOD AS COUNTER AND AVOID EVALUATING  *
//*  DAY CHANGE ETC.  *
//*---*
//ST002EXEC  PGM=SORT
//SYSOUTDD   SYSOUT=*
//SORTIN   DD DSN=,DISP=(OLD,DELETE)
//SORTOUT  DD SYSOUT=*
//SYSIN DD   *
 INREC FIELDS=(1,4,SEQNUM
  5,8, BASE TS
  1,4,BI,MUL,+4096,ADD,5,8,BI,TO=BI,LENGTH=8,  ADD SEQNUM US TO BASE TS
  ((1,4,BI,MUL,+4096,ADD,5,8,BI),  CALC REMAINDER US*1
   MOD,+4096),DIV,+4096,TO=BI,LENGTH=4)
 SORT FIELDS=COPY
 OUTREC FIELDS=(C'',
   13,8,TC4,
   21,4,BI,EDIT=())
/*





Il giorno mer 31 gen 2024 alle ore 20:10 Billy Ashton <
bill00ash...@gmail.com> ha scritto:

> Thanks so much for this, Kolusu. I was hoping to find a way to get a new
> time on every record, but I think I can work with this and the following
> sequence number.
>
> I hope you have a great day today!
>
> Thank you and best regards,
> Billy Ashton
>
>
> -- Original Message --
> From "Sri h Kolusu" 
> To IBM-MAIN@listserv.ua.edu
> Date 1/31/2024 1:32:23 PM
> Subject Re: Generating output in SORT with a time value
>
> >>>  Hi there Kolusu, I messed up my format, and should have had colons
> instead of commas.
> >
> >Billy,
> >
> >Apart from usage of Colons, did you realize that you have OVERLAPPING
> data?
> >
> >At position 82 you wanted to write 82:C'This is my data field 3',  and
> then at position 102 you wanted to write this 102:C'AZ',
> >
> >The length of the text at 82 is 23 bytes long.
> >
> >+1+2
>

Re: Generating output in SORT with a time value

[Massimo Biancucci]

Billy,

as Sri said there's no way to ask SORT tools to put a different TS on every
record.
Hereby a "try" to start with an arbitrary TS (in TOD format to generate as
you like) and then generate a number of consecutive TS part (HHMMSSuu)
with a distance of 1 us.

It's a two steps and probably can be shrinked into one.
Consider it as an idea to further develop.
This doesn't guarantee any uniqueness between two different runs or single
run generating more than "1 day" number of records.
Hope this helps.
Max

//*---*
//* LET'S GENERATE A ARBITRARY NUMBERS OF ROWS WITH THE SEQNUM IN *
//* BINARY FORMAT AND A FIX TOD. LET'S CALL IT BASE TOD.  *
//*---*
//ST001EXEC  PGM=SORT
//SYSOUTDD   SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD DSN=&,DISP=(,PASS),SPACE=(TRK,(10,10),RLSE)
//SYSIN DD   *
 SORT FIELDS=COPY
 OUTFIL REPEAT=10,
  BUILD=(SEQNUM,4,BI,START=0,X'DE956F5F25C18000')
*2024-01-31-23.59.59.999000
* BUILD=(SEQNUM,4,BI,X'DE9542AB0E80')
*2024-01-31-20.40.00.00
/*
//*---*
//* TOD IS A 8 BYTES BINARY WHERE BIT 51 IS 1 US  *
//* IT MEANS YOU HAVE TO DIVIDE TOD BY 4096 TO CALC NUM OF US *
//* SO:   *
//*---*
//* +1291714560   1 YEAR  *
//* +3538944  1 DAY   *
//* +147456   1 HOUR  *
//* +24576000 1 MINUTE*
//* +409600   1 SECOND*
//* +4096 1 US*
//*---*
//* SORT CAN TRANSFORM TOD TO DATE OR TIME.   *
//* FOR TIME THE MAXIMUM PRECISION IS HUNDREDTHS SECOND.  *
//* WE NEED TO CALCULATE THE LAST 4 DIGITS OF MICROSECONDS.   *
//*---*
//* FIRST ADD SEQNUM US TO THE BASE TOD.  *
//* EVERY RECORD WILL HAVE ONE US MORE.   *
//* LET'S PRINT TOD AS TC4: HHMMSSHH. *
//* LET'S CALC LAST 4 DIGITS AS REMAINDER OF TOD BY 4096. *
//* THIS IS THE VALUE IN TOD US UNITS. DIVIDE IT BY 4096 TO GET US.   *
//*---*
//* WE USE THIS MECHANISM TO USE TOD AS COUNTER AND AVOID EVALUATING  *
//*  DAY CHANGE ETC.  *
//*---*
//ST002EXEC  PGM=SORT
//SYSOUTDD   SYSOUT=*
//SORTIN   DD DSN=,DISP=(OLD,DELETE)
//SORTOUT  DD SYSOUT=*
//SYSIN DD   *
 INREC FIELDS=(1,4,SEQNUM
  5,8, BASE TS
  1,4,BI,MUL,+4096,ADD,5,8,BI,TO=BI,LENGTH=8,  ADD SEQNUM US TO BASE TS
  ((1,4,BI,MUL,+4096,ADD,5,8,BI),  CALC REMAINDER US*1
   MOD,+4096),DIV,+4096,TO=BI,LENGTH=4)
 SORT FIELDS=COPY
 OUTREC FIELDS=(C'',
   13,8,TC4,
   21,4,BI,EDIT=())
/*





Il giorno mer 31 gen 2024 alle ore 20:10 Billy Ashton <
bill00ash...@gmail.com> ha scritto:

> Thanks so much for this, Kolusu. I was hoping to find a way to get a new
> time on every record, but I think I can work with this and the following
> sequence number.
>
> I hope you have a great day today!
>
> Thank you and best regards,
> Billy Ashton
>
>
> -- Original Message --
> From "Sri h Kolusu" 
> To IBM-MAIN@listserv.ua.edu
> Date 1/31/2024 1:32:23 PM
> Subject Re: Generating output in SORT with a time value
>
> >>>  Hi there Kolusu, I messed up my format, and should have had colons
> instead of commas.
> >
> >Billy,
> >
> >Apart from usage of Colons, did you realize that you have OVERLAPPING
> data?
> >
> >At position 82 you wanted to write 82:C'This is my data field 3',  and
> then at position 102 you wanted to write this 102:C'AZ',
> >
> >The length of the text at 82 is 23 bytes long.
> >
> >+1+2
> >THIS IS MY DATA FIELD 3
> >
> >You want to write 23 bytes into a 20-byte length which is NOT possible.
> >
> >>>  I checked the APG, and could not find a way to use only the timestamp.
> >
> >Not every user requirement in the world can be documented.  You get the
> timestamp and strip of the values and ge

Re: Generating output in SORT with a time value

[Billy Ashton]

Thanks so much for this, Kolusu. I was hoping to find a way to get a new 
time on every record, but I think I can work with this and the following 
sequence number.


I hope you have a great day today!

Thank you and best regards,
Billy Ashton


-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 1/31/2024 1:32:23 PM
Subject Re: Generating output in SORT with a time value


 Hi there Kolusu, I messed up my format, and should have had colons instead of 
commas.


Billy,

Apart from usage of Colons, did you realize that you have OVERLAPPING data?

At position 82 you wanted to write 82:C'This is my data field 3',  and then at 
position 102 you wanted to write this 102:C'AZ',

The length of the text at 82 is 23 bytes long.

+1+2
THIS IS MY DATA FIELD 3

You want to write 23 bytes into a 20-byte length which is NOT possible.


 I checked the APG, and could not find a way to use only the timestamp.


Not every user requirement in the world can be documented.  You get the 
timestamp and strip of the values and get the desired value.

btw I hope you do realize that TIMESTAMP is got ONCE and since you are writing 
1500 records, every record will have the SAME exact timestamp.   The Timestamp 
is NOT changed for every record.

For example, this is the time stamp

+1+2+---
-mm-dd-hh.mm.ss.nn
2024-01-31-11.05.40.559424

Now you want to remove the separators and have the time as 110540559424.

That value will remain the SAME value for all the 1500 records.

If you want the same time for all the records, then here is the JCL that would 
get you the desired results ( I fixed the overlapping fields )

2 lines(see the lines in bold) is what is required to get the time without 
separators.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INREC BUILD=(DATE5)
  OUTFIL REPEAT=1500,
  BUILD=(01:C'TLX1',
 05:12,15,UFF,M11,LENGTH=12,
 17:SEQNUM,5,ZD,START=10001,INCR=3,
 22:C'THIS IS MY DATA FIELD 1',
 52:C'THIS IS MY DATA FIELD 2',
 82:C'THIS IS MY DATA FIELD 3',
112:C'AZ',
114:C'1')
/*
Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Generating output in SORT with a time value

[Sri h Kolusu]

>> Hi there Kolusu, I messed up my format, and should have had colons instead 
>> of commas.

Billy,

Apart from usage of Colons, did you realize that you have OVERLAPPING data?

At position 82 you wanted to write 82:C'This is my data field 3',  and then at 
position 102 you wanted to write this 102:C'AZ',

The length of the text at 82 is 23 bytes long.

+1+2
THIS IS MY DATA FIELD 3

You want to write 23 bytes into a 20-byte length which is NOT possible.

>> I checked the APG, and could not find a way to use only the timestamp.

Not every user requirement in the world can be documented.  You get the 
timestamp and strip of the values and get the desired value.

btw I hope you do realize that TIMESTAMP is got ONCE and since you are writing 
1500 records, every record will have the SAME exact timestamp.   The Timestamp 
is NOT changed for every record.

For example, this is the time stamp

+1+2+---
-mm-dd-hh.mm.ss.nn
2024-01-31-11.05.40.559424

Now you want to remove the separators and have the time as 110540559424.

That value will remain the SAME value for all the 1500 records.

If you want the same time for all the records, then here is the JCL that would 
get you the desired results ( I fixed the overlapping fields )

2 lines(see the lines in bold) is what is required to get the time without 
separators.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INREC BUILD=(DATE5)
  OUTFIL REPEAT=1500,
  BUILD=(01:C'TLX1',
 05:12,15,UFF,M11,LENGTH=12,
 17:SEQNUM,5,ZD,START=10001,INCR=3,
 22:C'THIS IS MY DATA FIELD 1',
 52:C'THIS IS MY DATA FIELD 2',
 82:C'THIS IS MY DATA FIELD 3',
112:C'AZ',
114:C'1')
/*
Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Generating output in SORT with a time value

[Billy Ashton]

Hi there Kolusu, I messed up my format, and should have had colons 
instead of commas. Here is what I should have written:

OUTFIL   REPEAT=1500
BUILD=(1:C'TLX1',
5:SEQNUM,12,ZD,START=12345600,INCR=1,  <<==This is the 
one to change from a number to a dynamic timestamp

   17:SEQNUM,5,ZD,START=10001,INCR=3
   22:C'This is my data field 1',
   52:C'This is my data field 2' ,
   82:C'This is my data field 3',
  102:C'AZ',
  104:C'1')

I checked the APG, and could not find a way to use only the timestamp. 
What I want in column 5 is a 12 numeral value (let's say 125110104322) 
that represents the HHMMSSmm for each record as it is built. I have 
seen examples using a static date for the entire execution, but I am 
hoping for multiple milliseconds to show up in these records.


Thank you and best regards,
Billy Ashton


-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 1/31/2024 12:06:40 PM
Subject Re: Generating output in SORT with a time value


 We have this so far:

OUTFIL   REPEAT=1500
BUILD=(1,C'TLX1',
5,SEQNUM,12,ZD,START=12345600,INCR=1,
   17,SEQNUM,5,ZD,START=10001,INCR=3
   22,C'This is my data field 1',
   52,C'This is my data field 2' ,
   82,C'This is my data field 3',
  102,C'AZ',
  104,C'1')


Billy,

Looks like you created your own syntax as that is NOT a valid for both DFSORT 
and Syncsort.


 IS there a way I can say "In column 5, put the right-now-time-millisecond value as 
HHMMSSmm". We will create multiple files at different times for our test, which 
is why we want to have the real time instead of a generated number.


No idea, what column 5 is.   If you want a 26 byte date of the format 
C'-mm-dd-hh.mm.ss.nn', then use DATE5. Look at examples in the DFSORT 
Application programming Guide.


Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Generating output in SORT with a time value

[Sri h Kolusu]

>> We have this so far:
OUTFIL   REPEAT=1500
BUILD=(1,C'TLX1',
5,SEQNUM,12,ZD,START=12345600,INCR=1,
   17,SEQNUM,5,ZD,START=10001,INCR=3
   22,C'This is my data field 1',
   52,C'This is my data field 2' ,
   82,C'This is my data field 3',
  102,C'AZ',
  104,C'1')


Billy,

Looks like you created your own syntax as that is NOT a valid for both DFSORT 
and Syncsort.

>> IS there a way I can say "In column 5, put the right-now-time-millisecond 
>> value as HHMMSSmm". We will create multiple files at different times for 
>> our test, which is why we want to have the real time instead of a generated 
>> number.

No idea, what column 5 is.   If you want a 26 byte date of the format 
C'-mm-dd-hh.mm.ss.nn', then use DATE5. Look at examples in the DFSORT 
Application programming Guide.


Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Generating output in SORT with a time value

[Billy Ashton]

Hey everyone!
I have a user who is creating some test data to load into VSAM using 
DFSORT, and he wants to create part of the key using the current 
millisecond timestamp. IS there a way I can say "In column 5, put the 
right-now-time-millisecond value as HHMMSSmm". We will create 
multiple files at different times for our test, which is why we want to 
have the real time instead of a generated number.


We have this so far:
OUTFIL   REPEAT=1500
BUILD=(1,C'TLX1',
   5,SEQNUM,12,ZD,START=12345600,INCR=1,
  17,SEQNUM,5,ZD,START=10001,INCR=3
  22,C'This is my data field 1',
  52,C'This is my data field 2' ,
  82,C'This is my data field 3',
 102,C'AZ',
 104,C'1')

Thank you and best regards,
Billy Ashton

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Wayne Bickerdike]

Agree, however, it took me less than 2 minutes to add the code for FIELD
card.

I was responding to the original, however, here's the extra code,

OUT2.I = 'RPT FIELD,TABLE,'||Dbname|| VERSION
SAY REC.I
SAY OUT.I
SAY OUT2.I
D
"EXECIO * DISKW  OFILE(STEM OUT. FINIS "
"EXECIO * DISKW  OFILE(STEM OUT2. FINIS "

Adding a return code for an empty file is also trivial.





On Fri, Nov 10, 2023 at 2:28 PM Sri h Kolusu  wrote:

> >> I tried this in REXX, more familiar for me than DFSORT.
>
> Wayne,
>
> Thanks for the REXX code, however, please note that there is additional
> processing.  OP wants to generate 2 different command files and make a copy
> of the input along with setting a return code if the input file is empty,
> so that they can skip further processing.
>
> Rexx is good at some tasks, but it is not good at I/O when compared to
> DFSORT.
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>


-- 
Wayne V. Bickerdike

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Sri h Kolusu]

>> I tried this in REXX, more familiar for me than DFSORT.

Wayne,

Thanks for the REXX code, however, please note that there is additional 
processing.  OP wants to generate 2 different command files and make a copy of 
the input along with setting a return code if the input file is empty, so that 
they can skip further processing.

Rexx is good at some tasks, but it is not good at I/O when compared to DFSORT. 

Thanks,
Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Wayne Bickerdike]

I would probably use a 26 char ALPHA string and index into that to
avoid the stem variable with the hard coded A to F.

or a Hex to Char based on the suffix value

Been a while since I saw Datacom/DB dictionary output :)

On Fri, Nov 10, 2023 at 1:13 PM Wayne Bickerdike  wrote:

> I tried this in REXX, more familiar for me than DFSORT.
>
> My code:
>
> Stor = ''
> ALPHA.1 = 'A'
> ALPHA.2 = 'B'
> ALPHA.3 = 'C'
> ALPHA.4 = 'D'
> ALPHA.5 = 'E'
> ALPHA.6 = 'F'
> "ALLOC F(IFILE) DA('BDB204.OUTLIST(FROM)') SHR REUSE   "
> "ALLOC F(OFILE) DA('BDB204.OUTLIST(TO)') SHR REUSE   "
>   "EXECIO * DISKR  IFILE(STEM REC. FINIS "
> Suffix = 0
> DO I = 1 TO REC.0
>   Parse Value REC.I ,
>   with DBID . . WORD1 Dbname VERSION .
> If POS(Dbname,Stor) > 0 Then Suffix = Suffix + 1
> Else Suffix = 1
>   Stor = Stor || Dbname
>   OUT.I = 'UTL COPY,TABLE,'||Dbname|| ,
>   VERSION || ','|| WORD1 || RIGHT('000' ||DBID,4) || ALPHA.Suffix
>   SAY REC.I
>   SAY OUT.I
> END
>   "EXECIO * DISKW  OFILE(STEM OUT. FINIS "
>
> 400  B400-DATABASE 1001 BRN   B400-BRANCH-BLDG (1001)
> T
> UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
>
> 400  B400-DATABASE3 HST   B400-TRANHIST  (0003) H
>
> UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
>
> 400  B400-DATABASE4 HST   B400-TRANHIST  (0004) H
>
> UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
>
> 400  B400-DATABASE5 HST   B400-TRANHIST  (0005) H
>
> UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C
>
> ***
>
>
> On Fri, Nov 10, 2023 at 4:56 AM Don Johnson <
> 02ee771a0785-dmarc-requ...@listserv.ua.edu> wrote:
>
>> Kolusu, thanks once again! This is now complete and working as I had
>> hoped!
>>
>> Have a good evening!
>> Don
>>
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>
>
>
> --
> Wayne V. Bickerdike
>
>

-- 
Wayne V. Bickerdike

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Wayne Bickerdike]

I tried this in REXX, more familiar for me than DFSORT.

My code:

Stor = ''
ALPHA.1 = 'A'
ALPHA.2 = 'B'
ALPHA.3 = 'C'
ALPHA.4 = 'D'
ALPHA.5 = 'E'
ALPHA.6 = 'F'
"ALLOC F(IFILE) DA('BDB204.OUTLIST(FROM)') SHR REUSE   "
"ALLOC F(OFILE) DA('BDB204.OUTLIST(TO)') SHR REUSE   "
  "EXECIO * DISKR  IFILE(STEM REC. FINIS "
Suffix = 0
DO I = 1 TO REC.0
  Parse Value REC.I ,
  with DBID . . WORD1 Dbname VERSION .
If POS(Dbname,Stor) > 0 Then Suffix = Suffix + 1
Else Suffix = 1
  Stor = Stor || Dbname
  OUT.I = 'UTL COPY,TABLE,'||Dbname|| ,
  VERSION || ','|| WORD1 || RIGHT('000' ||DBID,4) || ALPHA.Suffix
  SAY REC.I
  SAY OUT.I
END
  "EXECIO * DISKW  OFILE(STEM OUT. FINIS "

400  B400-DATABASE 1001 BRN   B400-BRANCH-BLDG (1001) T
UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
400  B400-DATABASE3 HST   B400-TRANHIST  (0003) H
UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
400  B400-DATABASE4 HST   B400-TRANHIST  (0004) H
UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
400  B400-DATABASE5 HST   B400-TRANHIST  (0005) H
UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C
***

On Fri, Nov 10, 2023 at 4:56 AM Don Johnson <
02ee771a0785-dmarc-requ...@listserv.ua.edu> wrote:

> Kolusu, thanks once again! This is now complete and working as I had hoped!
>
> Have a good evening!
> Don
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>


-- 
Wayne V. Bickerdike

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Don Johnson]

Kolusu, thanks once again! This is now complete and working as I had hoped!

Have a good evening!
Don

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Ituriel do Neto]

I think there is also another option, like below

 //STEP1  EXEC PGM=ICETOOL                                     
//TOOLMSG DD SYSOUT=*                                          
//DFSMSG  DD SYSOUT=*                                          
//IN      DD DISP=SHR,DSN=YOUR.DATASET 
//TOOLIN  DD *                                                 
* SET RC=12 IF THE 'IN' DATA SET IS EMPTY, OR                  
* SET RC=0 IF THE 'IN' DATA SET IS NOT EMPTY                   
 COUNT FROM(IN) EMPTY                                          
/*                                                             


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer






Em quinta-feira, 9 de novembro de 2023 às 13:18:44 BRT, Sri h Kolusu 
 escreveu: 





>> If the input file has no records to process, I would like to set a return 
>> code so I can bypass further processing. Is there a way that I can tweak 
>> this SORT input that you gave me to know if the SORTCMD1 file is empty, and 
>> set a CC for the step?

Don,

Yes, we can set a return code if the file is empty.  Cameron has the right idea 
of using NULLOUT, however it will NOT work in your case as you are using OUTFIL 
processing.  So, the parm you need to use is NULLOFL.

So, change this statement

OUTFIL BUILD=(1,133)

To

OUTFIL BUILD=(1,133),NULLOFL=RC4

This will set RC=4 when the input file is EMPTY.

Thanks,
Kolusu
DFSORT Development
IBM Corporation




--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Sri h Kolusu]

>> If the input file has no records to process, I would like to set a return 
>> code so I can bypass further processing. Is there a way that I can tweak 
>> this SORT input that you gave me to know if the SORTCMD1 file is empty, and 
>> set a CC for the step?

Don,

Yes, we can set a return code if the file is empty.  Cameron has the right idea 
of using NULLOUT, however it will NOT work in your case as you are using OUTFIL 
processing.   So, the parm you need to use is NULLOFL.

So, change this statement

OUTFIL BUILD=(1,133)

To

OUTFIL BUILD=(1,133),NULLOFL=RC4

This will set RC=4 when the input file is EMPTY.

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Cameron Conacher]

I believe that NULLOUT might help you here. (OPTION Statement)

Temporarily overrides the NULLOUT installation option, which specifies the 
action to be taken by DFSORT when there are no records for the SORTOUT data 
set, as indicated by an OUT count of 0 in message ICE054I RC0 specifies that 
DFSORT should issue message ICE173I, set a return code of 0, and continue 
processing when there are no records for the SORTOUT data set. RC4 specifies 
that DFSORT should issue message ICE173I, set a return code of 4, and continue 
processing when there are no records for the SORTOUT data set. RC16 specifies 
that DFSORT should issue message ICE206A, terminate, and give a return code of 
16 when there are no records for the SORTOUT data set.


Thanks

…….Cameron

From: IBM Mainframe Discussion List  On Behalf Of Don 
Johnson
Sent: Thursday, November 9, 2023 10:48 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT Help with JFY and SQZ

Hi Kolusu, I have one question about a situation that just came up. If the 
input file has no records to process, I would like to set a return code so I 
can bypass further processing. Is there a way that I can tweak this SORT input 
that you


Hi Kolusu, I have one question about a situation that just came up.



If the input file has no records to process, I would like to set a return code 
so I can bypass further processing. Is there a way that I can tweak this SORT 
input that you gave me to know if the SORTCMD1 file is empty, and set a CC for 
the step?



Don



--

For IBM-MAIN subscribe / signoff / archive access instructions,

send email to lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with 
the message: INFO IBM-MAIN


American Express made the following annotations

This e-mail was sent to you by a representative of Amex Bank of Canada, P.O. 
Box 3204, Station "F", Toronto, ON, M1W 3W7, www.americanexpress.ca. If you no 
longer wish to receive these e-mails, please notify the sender by reply e-mail.

This e-mail is solely for the intended recipient and may contain confidential 
or privileged information. If you are not the intended recipient, any 
disclosure, copying, use, or distribution of the information included in this 
e-mail is prohibited. If you have received this e-mail in error, please notify 
the sender by reply e-mail and immediately and permanently delete this e-mail 
and any attachments. Thank you.

American Express a fait les remarques suivantes
Ce courriel vous a été envoyé par un représentant de la Banque Amex du Canada, 
C.P. 3204, succursale F, Toronto (Ontario) M1W 3W7, www.americanexpress.ca. Si, 
par la suite, vous ne souhaitez plus recevoir ces courriels, veuillez en aviser 
les expéditeurs par courriel.

Ce courriel est réservé au seul destinataire indiqué et peut renfermer des 
renseignements confidentiels et privilégiés. Si vous n’êtes pas le destinataire 
prévu, toute divulgation, duplication, utilisation ou distribution du courriel 
est interdite. Si vous avez reçu ce courriel par erreur, veuillez en aviser 
l’expéditeur par courriel et détruire immédiatement le courriel et toute pièce 
jointe. Merci.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Don Johnson]

Hi Kolusu, I have one question about a situation that just came up.

If the input file has no records to process, I would like to set a return code 
so I can bypass further processing. Is there a way that I can tweak this SORT 
input that you gave me to know if the SORTCMD1 file is empty, and set a CC for 
the step?

Don

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Don Johnson]

Kolusu, thanks...this gave me just what I needed in those files!

Have a wonder-filled day!
Don

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Sri h Kolusu]

>> In total, I am trying to create three output files:
1. The command file with UTL COPY (which I have) 2. The command file with RPT 
FIELD 3. The original input file.

Don,

It is quite simple.  Here is a sample job that will give you the desired 
results.  I assumed that your input has an LRECL=133 and RECM=FB.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DISP=SHR,DSN=Your.Input.LRECL.133.FB.File
//SORTOUT  DD SYSOUT=*
//SORTCMD1 DD SYSOUT=*
//SORTCMD2 DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INCLUDE COND=(1,1,CH,EQ,C'>')
  INREC OVERLAY=(134:55,32,SQZ=(SHIFT=LEFT),
 88,06,
 C',',
 49,03,
 04,04,UFF,EDIT=(),
 SEQNUM,1,BI,START=193,RESTART=(55,32))

  OUTFIL BUILD=(1,133)

  OUTFIL FNAMES=SORTCMD1,
 BUILD=(134,47,SQZ=(SHIFT=LEFT,
 LEAD=C'-UTL COPY,TABLE,',
   LENGTH=80))

  OUTFIL FNAMES=SORTCMD2,
 BUILD=(134,38,SQZ=(SHIFT=LEFT,
 LEAD=C'RPT FIELD,TABLE,',
   LENGTH=80))

/*

SORTOUT will have a copy of the INPUT file as is.

SORTCMD1 file will have.

-UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
-UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
-UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
-UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C

SORTCMD2 file will have.

RPT FIELD,TABLE,B400-BRANCH-BLDG(1001)
RPT FIELD,TABLE,B400-TRANHIST(0003)
RPT FIELD,TABLE,B400-TRANHIST(0004)
RPT FIELD,TABLE,B400-TRANHIST(0005)

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Don Johnson]

Well, I found that I could not release 2 records into the Copy from INREC. So 
here is what I did to produce all the output. Kolusu, if you have an easier 
way, I would be happy to learn and adapt!

This is what I did to get what I needed:

//SYSINDD  * 
  OPTION COPY
  INREC IFOUTLEN=260,IFTHEN=(WHEN=(1,1,CH,EQ,C'>'),  
BUILD=(01,100,   
   55,32,SQZ=(SHIFT=LEFT),   
   88,06,
   C',', 
   49,03,
   04,04,UFF,EDIT=(),
   SEQNUM,1,BI,START=193,RESTART=(55,32),
   180:X,
   C'>', 
   55,32,SQZ=(SHIFT=LEFT),   
   88,06,
   260:X))   
 
  OUTFIL FNAMES=SORTOUT,BUILD=(01,100)   
 
  OUTFIL FNAMES=SORTCMD1,INCLUDE=(1,1,CH,EQ,C'>'),   
 BUILD=(101,79,SQZ=(SHIFT=LEFT,  
   LEAD=C'-UTL COPY,TABLE,'))
 
  OUTFIL FNAMES=SORTCMD2,INCLUDE=(1,1,CH,EQ,C'>'),   
 BUILD=(181,79,SQZ=(SHIFT=LEFT,  
   LEAD=C'-RPT FIELD,TABLE,'))   


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Don Johnson]

Thanks for the tip, Kolusu; it worked great!

Now, I was trying to create a second command file, without that last comma and 
field. For example, the statements you gave me created
-UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A

and for the second file, I need
-RPT FIELD,TABLE,B400-TRANHIST(0003)

How easy is it to omit the last 9 chars (the comma plus the HST0400A)? I was 
thinking that I could use a "1" and "2" indicator to create the Build I need, 
and then only select the OUTFIL/OUTREC for the command file I am building - is 
that the best approach (every selected record will create both command files)?

And finally, I had wanted to print out the report in its entirety that I use 
for SORTIN. I tried to add a simple OUTFILE for SORTOUT, but it only had the 
reformatted input. Is there a way to do that here, or will I need a separate 
Gener step?

In total, I am trying to create three output files:
1. The command file with UTL COPY (which I have)
2. The command file with RPT FIELD 
3. The original input file.

Don

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Jay Maynard]

I'm getting the distinct impression that DFSORT is becoming a Swiss Army
knife for the systems programmer, much like a SAS DATA step is.

On Wed, Nov 8, 2023 at 9:04 AM Sri h Kolusu  wrote:

> >> Here is my code...can someone help me to get rid of the spaces in the
> name, before the ( value?
>
>
> Don,
>
> You are complicating a simple request.  First build Only the fields you
> need, and you can add the static characters using LEAD on the SQZ operator.
>
> Also, you don't have to get a seqnum in zd format and then convert that
> into a Alphabet, you can directly use a BINARY format with sequence
> starting at 193 and it would be the EBCDIC characters.
>
> //STEP0100 EXEC PGM=SORT
> //SYSOUT   DD SYSOUT=*
> //SORTIN   DD DISP=SHR,DSN=Your.Input.FB.file
> //SORTOUT  DD SYSOUT=*
> //SYSINDD *
>   OPTION COPY
>   INCLUDE COND=(1,1,CH,EQ,C'>')
>
>   INREC BUILD=(55,32,SQZ=(SHIFT=LEFT),
>88,06,
>C',',
>49,03,
>04,04,UFF,EDIT=(),
>SEQNUM,1,BI,START=193,RESTART=(55,32),
>80:X)
>
>   OUTREC BUILD=(01,80,SQZ=(SHIFT=LEFT,
>LEAD=C'-UTL COPY,TABLE,'))
> /*
>
> The output from this is
>
> -UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
> -UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
> -UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
> -UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C
>
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>


-- 
Jay Maynard

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Sri h Kolusu]

>> Here is my code...can someone help me to get rid of the spaces in the name, 
>> before the ( value?


Don,

You are complicating a simple request.  First build Only the fields you need, 
and you can add the static characters using LEAD on the SQZ operator.

Also, you don't have to get a seqnum in zd format and then convert that into a 
Alphabet, you can directly use a BINARY format with sequence starting at 193 
and it would be the EBCDIC characters.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DISP=SHR,DSN=Your.Input.FB.file
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INCLUDE COND=(1,1,CH,EQ,C'>')

  INREC BUILD=(55,32,SQZ=(SHIFT=LEFT),
   88,06,
   C',',
   49,03,
   04,04,UFF,EDIT=(),
   SEQNUM,1,BI,START=193,RESTART=(55,32),
   80:X)

  OUTREC BUILD=(01,80,SQZ=(SHIFT=LEFT,
   LEAD=C'-UTL COPY,TABLE,'))
/*

The output from this is

-UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
-UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
-UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
-UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C


Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT Help with JFY and SQZ

[Bob Bridges]

Since listserv emails are always plain-text (well, at least in my experience), 
there's no way for you to control the font used at the other end.  All you can 
send are the text characters; the recipient controls what font is used for 
plain-text emails.

For just the reasons you're thinking of I tell my client to display plain-text 
emails in a fixed-spacing font*, so I looked at the below and thought "what 
formatting issues?  Looks fine to me".  I don't understand why everyone doesn't 
do that.

* It's Consolas, because it's the only one I see that slashes zeroes.

---
Bob Bridges, robhbrid...@gmail.com, cell 336 382-7313

/* Science is a way of trying not to fool yourself.  -Richard Feynman */

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Don 
Johnson
Sent: Wednesday, November 8, 2023 09:04

Good morning! I am trying to do what I thought was a simple SORT Copy function 
to create some commands from an input file, but I am being thwarted by embedded 
blanks in my line. Here is the input (Sorry for the formatting issues, I am not 
sure how to post fixed-space text): 

+1+2+3+4+5+6+7+8+9+-
>   400  B400-DATABASE 1001 BRN   B400-BRANCH-BLDG  
>(1001) T
>   400  B400-DATABASE3 HST   B400-TRANHIST 
>(0003) H 
>   400  B400-DATABASE4 HST   B400-TRANHIST 
>(0004) H 
>   400  B400-DATABASE5 HST   B400-TRANHIST 
>(0005) H 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SORT Help with JFY and SQZ

[Don Johnson]

Good morning! I am trying to do what I thought was a simple SORT Copy function 
to create some commands from an input file, but I am being thwarted by embedded 
blanks in my line. Here is the input (Sorry for the formatting issues, I am not 
sure how to post fixed-space text): 

+1+2+3+4+5+6+7+8+9+-
>   400  B400-DATABASE 1001 BRN   B400-BRANCH-BLDG  
>(1001) T
>   400  B400-DATABASE3 HST   B400-TRANHIST 
>(0003) H 
>   400  B400-DATABASE4 HST   B400-TRANHIST 
>(0004) H 
>   400  B400-DATABASE5 HST   B400-TRANHIST 
>(0005) H 

I want to add to this an Alphabetic sequence for duplicate names (the number of 
duplicates cannot be more than 10), but the only way I know to do this is to 
create a 2-digit numeric sequence, and then convert that to a single letter.

Here is the output I get currently:

+10...+20...+30...+40...+50...+60..
-UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
-UTL COPY,TABLE,B400-TRANHIST   (0003),HST0400A
-UTL COPY,TABLE,B400-TRANHIST   (0004),HST0400B
-UTL COPY,TABLE,B400-TRANHIST   (0005),HST0400C

But this is what I want

+10...+20...+30...+40...+50...+60..
-UTL COPY,TABLE,B400-BRANCH-BLDG(1001),BRN0400A
-UTL COPY,TABLE,B400-TRANHIST(0003),HST0400A
-UTL COPY,TABLE,B400-TRANHIST(0004),HST0400B
-UTL COPY,TABLE,B400-TRANHIST(0005),HST0400C

Here is my code...can someone help me to get rid of the spaces in the name, 
before the ( value?

//SYSINDD  *
  OPTION COPY   
  INREC IFTHEN=(WHEN=(1,1,CH,EQ,C'>'),  
BUILD=(01:01,01,
   02:49,03,
   05:55,32,
   40:88,06,
   50:04,04,UFF,M11,LENGTH=04,  
   60:SEQNUM,2,ZD,START=1,INCR=1,RESTART=(55,32)))  

  OUTFIL FNAMES=SORTCMD2,INCLUDE=(1,1,CH,EQ,C'>'),  
BUILD=(C'-UTL COPY,TABLE,', 
   05,32,   
   40,06,   
   C',',
   02,03,   
   50,04,   
   60,02,CHANGE=(1, 
   C'01',C'A',C'02',C'B',C'03',C'C',C'04',C'D', 
   C'05',C'E',C'06',C'F',C'07',C'G',C'08',C'H', 
   C'09',C'I',C'10',C'J'),NOMATCH=(C'$'),80:X)  

I tried to use 05,32,SQZ=(SHIFT=LEFT), and that may have worked, but then it 
just padded the rest of the 32 bytes with spaces, so it did nothing.

Thanks in advance!

Don Johnson

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Jack Zukt]

Hello Kolusu,

Thank you very much for your help.
I was not quite happy with the solution that I had come up with, as I was
reading the input file twice and that bothered me because that is not an
efficient solution.
You latest solution is much simpler and efficient. And much more elegant as
well.
Thank you once again for all your help
Best wishesJack

On Mon, 18 Sept 2023 at 16:50, Sri h Kolusu  wrote:

> >> Is there a way to get all the dataset profiles that have two different
> groups with the same access using SORT or ICETOOL but without using
> JOINKEYS, so that the input file would only needed to be read once, using
> SORTIN as the DDNAME?
>
> Jack,
>
> That is quite simple.  There is a lot of functionality that DFSORT can
> offer to get the desired results.
>
> So, if your intention is to find the COMMON datasets for ACC of GPRFCICS
> and GPRFSTG, here is a JCL that will give you the desired results.  You
> don't even have to filter out Type 0404 records separately as we can do it
> in a Single pass of data from the raw IRRDBU00 output.
>
> //STEP0100 EXEC PGM=SORT
> //SYSOUT   DD SYSOUT=*
> //SORTDIAG DD DUMMY
> //SORTIN   DD DISP=SHR,DSN=Your.raw.IRRDBU00.output
> //SORTOUT  DD SYSOUT=*
> //SYMNAMES DD *
> ** INPUT TYPE 0404 RECORD LAYOUT **
> TYP,05,04,CH
>T404,C'0404'
> DSN,10,44,CH
> GRP,62,08,SS
>STG,C'GPRFSTG'
>   CICS,C'GPRFCICS'
> ACC,71,08,CH
>   ALTER,C'ALTER'
>READ,C'READ'
> ** REFORMATTED RECORD LAYOUT **
> IN-RDW,01,04,BI
> IN-DSN,*,44,CH
> SKIP,1
> IN-GRP,*,08,CH
> SKIP,1
> IN-ACC,*,08,CH
> SKIP,1
> TP-ACC,*,08,CH
> /*
> //SYSIN DD *
>   OPTION VLSCMP
>   INCLUDE COND=(TYP,EQ,T404,AND,
> ACC,EQ,ALTER,AND,
>(GRP,EQ,STG,OR,
>     GRP,EQ,CICS))
>
>INREC BUILD=(IN-RDW,
> DSN,C'|',
> ACC,C'|',
> GRP,C'|')
>
>   SORT FIELDS=(IN-DSN,A,
>IN-ACC,A)
>
>   OUTREC IFTHEN=(WHEN=GROUP,KEYBEGIN=(IN-DSN),
>  PUSH=(TP-ACC:IN-ACC))
>
>   OUTFIL VTOF,
>   BUILD=(IN-DSN),
>   INCLUDE=(IN-ACC,EQ,STG,AND,
>TP-ACC,EQ,CICS)
> /*
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Sri h Kolusu]

>> Is there a way to get all the dataset profiles that have two different 
>> groups with the same access using SORT or ICETOOL but without using 
>> JOINKEYS, so that the input file would only needed to be read once, using 
>> SORTIN as the DDNAME?

Jack,

That is quite simple.  There is a lot of functionality that DFSORT can offer to 
get the desired results.

So, if your intention is to find the COMMON datasets for ACC of GPRFCICS and 
GPRFSTG, here is a JCL that will give you the desired results.  You don't even 
have to filter out Type 0404 records separately as we can do it in a Single 
pass of data from the raw IRRDBU00 output.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTDIAG DD DUMMY
//SORTIN   DD DISP=SHR,DSN=Your.raw.IRRDBU00.output
//SORTOUT  DD SYSOUT=*
//SYMNAMES DD *
** INPUT TYPE 0404 RECORD LAYOUT **
TYP,05,04,CH
   T404,C'0404'
DSN,10,44,CH
GRP,62,08,SS
   STG,C'GPRFSTG'
  CICS,C'GPRFCICS'
ACC,71,08,CH
  ALTER,C'ALTER'
   READ,C'READ'
** REFORMATTED RECORD LAYOUT **
IN-RDW,01,04,BI
IN-DSN,*,44,CH
SKIP,1
IN-GRP,*,08,CH
SKIP,1
IN-ACC,*,08,CH
SKIP,1
TP-ACC,*,08,CH
/*
//SYSIN DD *
  OPTION VLSCMP
  INCLUDE COND=(TYP,EQ,T404,AND,
ACC,EQ,ALTER,AND,
   (GRP,EQ,STG,OR,
GRP,EQ,CICS))

   INREC BUILD=(IN-RDW,
DSN,C'|',
ACC,C'|',
    GRP,C'|')

  SORT FIELDS=(IN-DSN,A,
   IN-ACC,A)

  OUTREC IFTHEN=(WHEN=GROUP,KEYBEGIN=(IN-DSN),
 PUSH=(TP-ACC:IN-ACC))

  OUTFIL VTOF,
  BUILD=(IN-DSN),
  INCLUDE=(IN-ACC,EQ,STG,AND,
   TP-ACC,EQ,CICS)
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Jack Zukt]

Hi Kolusu

I tried the JCL that you have sent.
It works, as expected, but maybe I was not quite clear in my last question.
Is there a way to get all the dataset profiles that have two different
groups with the same access using SORT or ICETOOL but without using
JOINKEYS, so that the input file would only needed to be read once, using
SORTIN as the DDNAME?
This is me being curious and trying to use a more efficient solution, if
that would be possible.I could not came up with any kind of SORT statements
to that effect.
Best regards
Jack


On Fri, 15 Sept 2023 at 18:33, Sri h Kolusu  wrote:

> >> Now, my question is, is there a way to do this with only one step and
> reading the input file only once?
>
>
> Jack,
>
> Here is 1 step JCL that would match the STG with CICS records the type
> 0404 records from the input file.  Note that we can additional
> functionality if you need after the match is done.
>
> //STEP0100 EXEC PGM=SORT
> //SYSOUT   DD SYSOUT=*
> //SORTDIAG DD DUMMY
> //SYMNAMES DD *
> ** INPUT TYPE 0404 RECORD LAYOUT **
> IN-RDW,01,04,BI
> DSN,10,44,CH
> GRP,62,08,CH
>STG,C'GPRFSTG'
>   CICS,C'GPRFCICS'
> ACC,71,08,CH
>   ALTER,C'ALTER'
>READ,C'READ'
>
> ** STG REFORMATTED RECORD LAYOUT **
> STG-RDW,01,04,BI
> STG-DSN,*,44,CH
> SKIP,1
> STG-GRP,*,08,CH
> SKIP,1
> STG-ACC,*,08,CH
> SKIP,1
>
> ** CICS REFORMATTED RECORD LAYOUT **
> CICS-RDW,01,04,BI
> CICS-DSN,*,44,CH
> SKIP,1
> CICS-GRP,*,08,CH
> SKIP,1
> CICS-ACC,*,08,CH
> SKIP,1
> /*
> //STG  DD DISP=SHR,DSN=
> //CICS DD DISP=SHR,DSN=
> //SORTOUT  DD SYSOUT=*
> //SYSIN DD *
>   JOINKEYS F1=STG,FIELDS=(STG-DSN,A,STG-ACC,A)
>   JOINKEYS F2=CICS,FIELDS=(CICS-DSN,A,CICS-ACC,A)
>   REFORMAT FIELDS=(F1:STG-DSN)
>   OPTION COPY
> /*
> //JNF1CNTL  DD *
>   OPTION VLSCMP
>   INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,STG)
>INREC BUILD=(IN-RDW,
> DSN,C'|',
> ACC,C'|',
> GRP,C'|')
> /*
> //JNF2CNTL  DD *
>   OPTION VLSCMP
>   INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,CICS)
>INREC BUILD=(IN-RDW,
> DSN,C'|',
> ACC,C'|',
> GRP,C'|')
> /*
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Sri h Kolusu]

>> Now, my question is, is there a way to do this with only one step and 
>> reading the input file only once?


Jack,

Here is 1 step JCL that would match the STG with CICS records the type 0404 
records from the input file.  Note that we can additional functionality if you 
need after the match is done.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTDIAG DD DUMMY
//SYMNAMES DD *
** INPUT TYPE 0404 RECORD LAYOUT **
IN-RDW,01,04,BI
DSN,10,44,CH
GRP,62,08,CH
   STG,C'GPRFSTG'
  CICS,C'GPRFCICS'
ACC,71,08,CH
  ALTER,C'ALTER'
   READ,C'READ'

** STG REFORMATTED RECORD LAYOUT **
STG-RDW,01,04,BI
STG-DSN,*,44,CH
SKIP,1
STG-GRP,*,08,CH
SKIP,1
STG-ACC,*,08,CH
SKIP,1

** CICS REFORMATTED RECORD LAYOUT **
CICS-RDW,01,04,BI
CICS-DSN,*,44,CH
SKIP,1
CICS-GRP,*,08,CH
SKIP,1
CICS-ACC,*,08,CH
SKIP,1
/*
//STG  DD DISP=SHR,DSN=
//CICS DD DISP=SHR,DSN=
//SORTOUT  DD SYSOUT=*
//SYSIN DD *
  JOINKEYS F1=STG,FIELDS=(STG-DSN,A,STG-ACC,A)
  JOINKEYS F2=CICS,FIELDS=(CICS-DSN,A,CICS-ACC,A)
  REFORMAT FIELDS=(F1:STG-DSN)
  OPTION COPY
/*
//JNF1CNTL  DD *
  OPTION VLSCMP
  INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,STG)
   INREC BUILD=(IN-RDW,
DSN,C'|',
ACC,C'|',
GRP,C'|')
/*
//JNF2CNTL  DD *
  OPTION VLSCMP
  INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,CICS)
   INREC BUILD=(IN-RDW,
DSN,C'|',
ACC,C'|',
GRP,C'|')
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Jack Zukt]

Hi Kolusu,

Thank you very much for your input.
The input file already is a subset of 0404 records, and the SUM FIELDS=NONE
was the culprit. Once again the trouble come from copying a previously used
JCL and not cleaning it properly.
Now, if you will not mind, I have another question regarding this JCL. As I
stated previously, I am trying to find out all the dataset profiles that
have group1 and group2 on its access list with ALTER authority.
I have come up with two different JCL two get the desired result.
The one I posted previously does a first sort to get the group1 and group2
records into two datasets, and then does a second step with the JOINKEYS
statement.

The second one has only one step, with JNF1CNTL and JNF2CNTL cards for
record selection:
//IN1   DD DISP=SHR,DSN=
//IN2   DD DISP=SHR,DSN=
//SYMNAMES DD  *
DSN,10,44,CH
GRP,62,08,CH
   STG,C'GPRFSTG'
  CICS,C'GPRFCICS'
ACC,71,08,CH
  ALTER,C'ALTER'
   READ,C'READ'
//SYSIN DD *
  JOINKEYS F1=IN1,FIELDS=(DSN,A,ACC,A)
  JOINKEYS F2=IN2,FIELDS=(DSN,A,ACC,A)
  REFORMAT FIELDS=(F1:DSN)
  OPTION COPY
  OUTFIL FNAMES=OUT,VTOF,BUILD=(1,44)
/*
//JNF1CNTL  DD *
  INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,STG)
/*
//JNF2CNTL  DD *
  INCLUDE COND=(ACC,EQ,ALTER,AND,GRP,EQ,CICS)
/*

This last one has only one step but it reads the same input file once for
each INx card.
The first one reads the input file only once, but it has two steps, and the
first step created files have to be read on the second step.
Now, my question is, is there a way to do this with only one step and
reading the input file only once?

Regards,
Jack


On Fri, 15 Sept 2023 at 15:30, Sri h Kolusu  wrote:

> >>> I am using for the INPUT a subset of a type 0404 records from a
> IRRDBU00 output file. In the first step I am crating two files with
>
> Jack,
>
> Not clear if you still need help , but the INCLUDE COND does not have the
> include for type 0404 records, unless you already filtered your input.
>
> Can you please run the following.
>
> //STEP0100 EXEC PGM=SORT
> //SYSOUT   DD SYSOUT=*
> //SYMNAMES DD  *
> TYP,05,04,CH
> DSN,10,44,CH
> GRP,62,08,CH
> ACC,71,08,CH
> //SORTIN   DD DISP=SHR,DSN=Your.RACF.IRRDBU00.file
> //SORTOUT  DD SYSOUT=*
> //SYSINDD *
>   OPTION VLSCMP
>   INCLUDE COND=(TYP,EQ,C'0404',AND,
> ACC,EQ,C'ALTER',AND,
>    (GRP,EQ,C'GPRFCICS',OR,
> GRP,EQ,C'GPRFSTG'))
>
>   SORT FIELDS=(DSN,A,ACC,A)
>
>   OUTFIL REMOVECC,VTOF,BUILD=(80X),
>   SECTIONS=(DSN,ACC,
>   TRAILER3=(DSN,C'|',
> ACC,C'|',
> GRP,C'|',
> COUNT,C'|'))
> /*
>
> Now check the results and look at the count column to see any value is
> greater than 1. If it has a value greater than 1 , then your SUM
> FIELDS=NONE will eliminate such records.
>
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Sri h Kolusu]

>>> I am using for the INPUT a subset of a type 0404 records from a  IRRDBU00 
>>> output file. In the first step I am crating two files with

Jack,

Not clear if you still need help , but the INCLUDE COND does not have the 
include for type 0404 records, unless you already filtered your input.

Can you please run the following.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SYMNAMES DD  *
TYP,05,04,CH
DSN,10,44,CH
GRP,62,08,CH
ACC,71,08,CH
//SORTIN   DD DISP=SHR,DSN=Your.RACF.IRRDBU00.file
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION VLSCMP
  INCLUDE COND=(TYP,EQ,C'0404',AND,
ACC,EQ,C'ALTER',AND,
   (GRP,EQ,C'GPRFCICS',OR,
GRP,EQ,C'GPRFSTG'))

  SORT FIELDS=(DSN,A,ACC,A)

  OUTFIL REMOVECC,VTOF,BUILD=(80X),
  SECTIONS=(DSN,ACC,
  TRAILER3=(DSN,C'|',
ACC,C'|',
GRP,C'|',
COUNT,C'|'))
/*

Now check the results and look at the count column to see any value is greater 
than 1. If it has a value greater than 1 , then your SUM FIELDS=NONE will 
eliminate such records.


Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT INCLUDE problem

[Jack Zukt]

Ok,
I found the problem (I think)
It was probably due to the "SUM FIELDS=NONE" statement
Best wishes
Jack

On Fri, 15 Sept 2023 at 10:42, Jack Zukt  wrote:

> Hi,
>
> I am trying to find a RACF dataset profile for which two different groups
> have ALTER access.
> I am using for the INPUT a subset of a type 0404 records from a IRRDBU00
> output file. In the first step I am crating two files with the dataset
> profiles in which the groups have ALTER authority:
>
> //SYMNAMES DD  *
> DSN,10,44,CH
> GRP,62,08,CH
> ACC,71,08,CH
> //SYSINDD  *
>   SORT FIELDS=(DSN,A,ACC,A)
>   SUM FIELDS=NONE
>   OPTION VLSHRT
>   INCLUDE COND=(ACC,EQ,C'ALTER   ',AND,
>(GRP,EQ,C'GPRFCICS',OR,
> GRP,EQ,C'GPRFSTG '))
>   OUTFIL FNAMES=OUT01,VTOF,
>   INCLUDE=(ACC,EQ,C'ALTER   ',AND,
> GRP,EQ,C'GPRFSTG '),
>   BUILD(DSN,
> ACC,
> GRP)
>   OUTFIL FNAMES=OUT02,VTOF,
>   INCLUDE=(ACC,EQ,C'ALTER   ',AND,
>GRP,EQ,C'GPRFCICS'),
>   BUILD(DSN,
> ACC,
> GRP)
> /*
>
> Now, for the OUT01 file, this SORT has 427 records. But, if I run the SORT
> with this statements:
>
> //SYMNAMES DD  *
> DSN,10,44,CH
> GRP,62,08,CH
> ACC,71,08,CH
> //SYSINDD  *
>   SORT FIELDS=(DSN,A,ACC,A)
>   SUM FIELDS=NONE
>   OPTION VLSHRT
>   INCLUDE COND=(ACC,EQ,C'ALTER   ',AND,
> GRP,EQ,C'GPRFSTG ')
>   OUTFIL FNAMES=OUT01,VTOF,
>   BUILD(DSN,
> ACC,
> GRP)
>
> the OUT01 file has 458 records.
>
> The OUT02 file has always 484 records on either type of sort.
> I only found out because the only dataset profile record for which both
> groups have access is not included on the first OUT01 file.
>
> Any ideas, please?
> I really cannot figure this one out.
> Thank you
> Best wishes
> Jack
>
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SORT INCLUDE problem

[Jack Zukt]

Hi,

I am trying to find a RACF dataset profile for which two different groups
have ALTER access.
I am using for the INPUT a subset of a type 0404 records from a IRRDBU00
output file. In the first step I am crating two files with the dataset
profiles in which the groups have ALTER authority:

//SYMNAMES DD  *
DSN,10,44,CH
GRP,62,08,CH
ACC,71,08,CH
//SYSINDD  *
  SORT FIELDS=(DSN,A,ACC,A)
  SUM FIELDS=NONE
  OPTION VLSHRT
  INCLUDE COND=(ACC,EQ,C'ALTER   ',AND,
   (GRP,EQ,C'GPRFCICS',OR,
GRP,EQ,C'GPRFSTG '))
  OUTFIL FNAMES=OUT01,VTOF,
  INCLUDE=(ACC,EQ,C'ALTER   ',AND,
GRP,EQ,C'GPRFSTG '),
  BUILD(DSN,
ACC,
GRP)
  OUTFIL FNAMES=OUT02,VTOF,
  INCLUDE=(ACC,EQ,C'ALTER   ',AND,
   GRP,EQ,C'GPRFCICS'),
  BUILD(DSN,
ACC,
GRP)
/*

Now, for the OUT01 file, this SORT has 427 records. But, if I run the SORT
with this statements:

//SYMNAMES DD  *
DSN,10,44,CH
GRP,62,08,CH
ACC,71,08,CH
//SYSINDD  *
  SORT FIELDS=(DSN,A,ACC,A)
  SUM FIELDS=NONE
  OPTION VLSHRT
  INCLUDE COND=(ACC,EQ,C'ALTER   ',AND,
GRP,EQ,C'GPRFSTG ')
  OUTFIL FNAMES=OUT01,VTOF,
  BUILD(DSN,
ACC,
GRP)

the OUT01 file has 458 records.

The OUT02 file has always 484 records on either type of sort.
I only found out because the only dataset profile record for which both
groups have access is not included on the first OUT01 file.

Any ideas, please?
I really cannot figure this one out.
Thank you
Best wishes
Jack

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Jon Perryman]

 > On Thursday, July 27, 2023 at 05:25:49 AM PDT, Jason Cai 
 >  wrote:
> The first question is about how to monitor and alert the system issues.

Hi Jason,

I don't think anyone analyzes messages on a daily/weekly/monthly basis because 
it is impractical. The majority of the thousands of unique message IDs (IBM and 
vendor) are rarely displayed. For instance, how often do you see JES2 
checkpoint delay?

Important messages do not scroll automatically off the operator screen and most 
automation products can detect non-scrollable messages. Write a rule or trap 
for non-scroll messages. In your situation, you have 3 scenarios to handle. 1. 
Messages to alert sysprogs, 2. Messages sysprogs want to ignore, 3. Messages 
that need to be placed into 1 or 2. For 3, your automation product can create a 
list to be reviewed. You should also consider whether sysprogs should be 
alerted to unclassed messages. 

As a new automation sysprog, I recommend that you watch for changed, added and 
deleted messages during upgrades and PTF apply. IBM and all reputable vendors 
provide this list thru SMP/E hold data and upgrade documentation. You need to 
review these messages to see if automation should be changed.

On Thursday, July 27, 2023 at 05:25:49 AM PDT, Jason Cai 
 wrote:  
 
 Dear Max

I am writing to you regarding two questions that I have.

The first question is about how to monitor and alert the system issues. We can 
monitor most of the issues and send alerts to our phones. However, when we 
cannot provide a complete list of msgids to the monitoring system, some 
critical alerts may be missed. How can we minimize the missing of critical 
alerts and also reduce the false alarms? This is our biggest challenge. We want 
to analyze the operlog every day to find out if there are any potential risks 
in the system and also to locate the problem when it occurs. I wonder how your 
shop handles this situation.

The second question is a simplified one. Can dfsort delete the last word if the 
first position is M and the last word is a number?

I appreciate your time and attention.

Sincerely,
Jason Cai

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
  

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Jason Cai]

Dear Max,

Thank you very much for your effort and help to provide me with a complete 
solution for my problem using SORT. That is exactly what I needed. I appreciate 
your expertise and generosity in sharing your sample JCL. It was very helpful 
and clear. I will use it as a reference and tailor it to fit my specific needs.

Thank you again for your assistance. You are very kind and professional.

Best Regards,

Jason Cai

==
Jason,

for you first question I could say experience.
All my customer sites have different "hot messages" to look at.
Others are only warning, others (the most of) are simply ignored.
I cannot simply share them.
I could give you some "generic hints" like:

   - JES Spool over x% then add a spool volume
   - IRA* messages that warn about critical memory issues
   - .

sorry for that.

About SORT, I can give you a sample to approximate what you need even
though it must be tailored to fit your big picture.
Assuming my input, SYSLOG (VBM), and no more than 28 words after the MSGID
and single word max 30 characters.
If some words are separated by more than one blank the multiple spaces will
be reduced to one.

I'm sure Sri can give you something better.

OPTION VLSCMP
 INCLUDE COND=(6,1,SS,EQ,C'M')
 INREC IFTHEN=(WHEN=INIT,
  PARSE=(%00=(ABSPOS=5,FIXLEN=59),
 %01=(ENDBEFR=C' ',FIXLEN=30,REPEAT=29)),
   BUILD=(1,4,%00,%01,%02,%03,%04,%05,%06,%07,%08,%09,
  %10,%11,%12,%13,%14,%15,%16,%17,%18,%19,
  %20,%21,%22,%23,%24,%25,%26,%27,%28,%29)),
IFTHEN=(WHEN=(904,3,CH,GE,C'000',AND,907,1,CH,EQ,C' '),
  OVERLAY=(904:30X)),
 IFTHEN=(WHEN=(874,3,CH,GE,C'000',AND,877,1,CH,EQ,C' '),
  OVERLAY=(874:30X)),
 IFTHEN=(WHEN=(844,3,CH,GE,C'000',AND,847,1,CH,EQ,C' '),
  OVERLAY=(844:30X)),
 IFTHEN=(WHEN=(814,3,CH,GE,C'000',AND,817,1,CH,EQ,C' '),
  OVERLAY=(814:30X)),
 IFTHEN=(WHEN=(784,3,CH,GE,C'000',AND,787,1,CH,EQ,C' '),
  OVERLAY=(784:30X)),
 IFTHEN=(WHEN=(754,3,CH,GE,C'000',AND,757,1,CH,EQ,C' '),
  OVERLAY=(754:30X)),
 IFTHEN=(WHEN=(724,3,CH,GE,C'000',AND,727,1,CH,EQ,C' '),
  OVERLAY=(724:30X)),
 IFTHEN=(WHEN=(694,3,CH,GE,C'000',AND,697,1,CH,EQ,C' '),
  OVERLAY=(694:30X)),
 IFTHEN=(WHEN=(664,3,CH,GE,C'000',AND,667,1,CH,EQ,C' '),
  OVERLAY=(664:30X)),
 IFTHEN=(WHEN=(634,3,CH,GE,C'000',AND,637,1,CH,EQ,C' '),
  OVERLAY=(634:30X)),
 IFTHEN=(WHEN=(604,3,CH,GE,C'000',AND,607,1,CH,EQ,C' '),
  OVERLAY=(604:30X)),
 IFTHEN=(WHEN=(574,3,CH,GE,C'000',AND,577,1,CH,EQ,C' '),
  OVERLAY=(574:30X)),
 IFTHEN=(WHEN=(544,3,CH,GE,C'000',AND,547,1,CH,EQ,C' '),
  OVERLAY=(544:30X)),
 IFTHEN=(WHEN=(514,3,CH,GE,C'000',AND,517,1,CH,EQ,C' '),
  OVERLAY=(514:30X)),
 IFTHEN=(WHEN=(484,3,CH,GE,C'000',AND,487,1,CH,EQ,C' '),
  OVERLAY=(484:30X)),
 IFTHEN=(WHEN=(454,3,CH,GE,C'000',AND,457,1,CH,EQ,C' '),
  OVERLAY=(454:30X)),
 IFTHEN=(WHEN=(424,3,CH,GE,C'000',AND,427,1,CH,EQ,C' '),
  OVERLAY=(424:30X)),
 IFTHEN=(WHEN=(394,3,CH,GE,C'000',AND,397,1,CH,EQ,C' '),
  OVERLAY=(394:30X)),
 IFTHEN=(WHEN=(364,3,CH,GE,C'000',AND,367,1,CH,EQ,C' '),
  OVERLAY=(364:30X)),
 IFTHEN=(WHEN=(334,3,CH,GE,C'000',AND,337,1,CH,EQ,C' '),
  OVERLAY=(334:30X)),
 IFTHEN=(WHEN=(304,3,CH,GE,C'000',AND,307,1,CH,EQ,C' '),
  OVERLAY=(304:30X)),
 IFTHEN=(WHEN=(274,3,CH,GE,C'000',AND,277,1,CH,EQ,C' '),
  OVERLAY=(274:30X)),
 IFTHEN=(WHEN=(244,3,CH,GE,C'000',AND,247,1,CH,EQ,C' '),
  OVERLAY=(244:30X)),
 IFTHEN=(WHEN=(214,3,CH,GE,C'000',AND,217,1,CH,EQ,C' '),
  OVERLAY=(214:30X)),
 IFTHEN=(WHEN=(184,3,CH,GE,C'000',AND,187,1,CH,EQ,C' '),
  OVERLAY=(184:30X)),
 IFTHEN=(WHEN=(154,3,CH,GE,C'000',AND,157,1,CH,EQ,C' '),
  OVERLAY=(154:30X)),
 IFTHEN=(WHEN=(124,3,CH,GE,C'000',AND,127,1,CH,EQ,C' '),
  OVERLAY=(124:30X)),
 IFTHEN=(WHEN=(094,3,CH,GE,C'000',AND,097,1,CH,EQ,C' '),
  OVERLAY=(094:30X)),
 IFTHEN=(WHEN=(064,3,CH,GE,C'000',AND,067,1,CH,EQ,C' '),
  OVERLAY=(064:30X))
 SORT FIELDS=COPY
 OUTFIL FNAMES=SORTOUT,
  OUTREC=(1,4,5,59,
  64,870,SQZ=(SHIFT=LEFT,MID=C' '))

Best regards.
Max

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Jason Cai]

Dear Kevin,

Thank you very much for your time and effort to help me find useful tools for 
log analysis. I appreciate your detailed and informative response. I will try 
each of the tools you suggested and see which one works best for me.

By the way, I have a question about IBM Z Operational Log Analytics and IBM Z 
Anomaly Analytics with Watson. Are these tools free or do they require a 
license fee? How can I get access to them if I want to try them out?


Thanks a lot!

Jason Cai
==
Found it.  The tool that should do what you want, with a bit of figuring out 
the syntax of the JCL, is 
https://github.com/IBM/IBM-Z-zOS/tree/main/zOS-Tools-and-Toys/msglg610.  Among 
other things, given a syslog, it will print out the number of times each 
message ID appears in the syslog.

Note that this is provided with absolutely no support or warranty or anything 
else from IBM.

And I would urge you to look at one of the other log monitoring tools, so 
you’re not having to do all the monitoring / analysis yourself.
--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock


From: IBM Mainframe Discussion List  on behalf of 
Kevin Mckenzie 
Date: Thursday, July 27, 2023 at 12:12 PM
To: IBM-MAIN@LISTSERV.UA.EDU 
Subject: [EXTERNAL] Re: Inquiry about extracting and counting msgid from 
operlog using sort program
Hi, Jason.  If all you’re looking to do is summarize log data, I can give you 
some options, at least.

There was/is a tool that Kevin Kelley, since retired, wrote to help clients do 
log analysis that would do exactly what you want, on platform, in batch; I’m 
trying to find it again.  It was available on testcase.boulder.ibm.com, and may 
well still be there in some subdirectory; I’ll spend a bit of time searching 
for it again.  Links to it have apparently been lost.  I believe it was 
mentioned in some Redbooks as well.  If I can find it, I’ll put it in the z/OS 
github repository.  It was intended to help clients do message suppression and 
the like.

I have a python library, zoslogs (https://pypi.org/project/zoslogs/ ), that I 
built to do very basic log parsing.  So if you wanted to, you could use it as a 
basis for something like what you’re trying to do; you at least wouldn’t have 
to worry about doing the initial parsing yourself.

There are some tools that might give you an easier interface to gather the 
syslog data, such as the z Common Data Provider, or the z/OSMF REST API.  I 
believe zoau does something similar.

There are also things like IBM Z Operational Log Analytics and IBM Z Anomaly 
Analytics with Watson that will do realtime/almost realtime analysis of the 
logs for you, and try to warn you about potential issues.

As others said, however, I don’t think using something like DFSORT will meet 
your needs, without a lot of work.  At the least I’d suggest using a 
pre-processing tool to handle the initial log parsing before you sort/summarize 
the data.
--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Kevin Mckenzie]

Found it.  The tool that should do what you want, with a bit of figuring out 
the syntax of the JCL, is 
https://github.com/IBM/IBM-Z-zOS/tree/main/zOS-Tools-and-Toys/msglg610.  Among 
other things, given a syslog, it will print out the number of times each 
message ID appears in the syslog.

Note that this is provided with absolutely no support or warranty or anything 
else from IBM.

And I would urge you to look at one of the other log monitoring tools, so 
you’re not having to do all the monitoring / analysis yourself.
--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock


From: IBM Mainframe Discussion List  on behalf of 
Kevin Mckenzie 
Date: Thursday, July 27, 2023 at 12:12 PM
To: IBM-MAIN@LISTSERV.UA.EDU 
Subject: [EXTERNAL] Re: Inquiry about extracting and counting msgid from 
operlog using sort program
Hi, Jason.  If all you’re looking to do is summarize log data, I can give you 
some options, at least.

There was/is a tool that Kevin Kelley, since retired, wrote to help clients do 
log analysis that would do exactly what you want, on platform, in batch; I’m 
trying to find it again.  It was available on testcase.boulder.ibm.com, and may 
well still be there in some subdirectory; I’ll spend a bit of time searching 
for it again.  Links to it have apparently been lost.  I believe it was 
mentioned in some Redbooks as well.  If I can find it, I’ll put it in the z/OS 
github repository.  It was intended to help clients do message suppression and 
the like.

I have a python library, zoslogs (https://pypi.org/project/zoslogs/ ), that I 
built to do very basic log parsing.  So if you wanted to, you could use it as a 
basis for something like what you’re trying to do; you at least wouldn’t have 
to worry about doing the initial parsing yourself.

There are some tools that might give you an easier interface to gather the 
syslog data, such as the z Common Data Provider, or the z/OSMF REST API.  I 
believe zoau does something similar.

There are also things like IBM Z Operational Log Analytics and IBM Z Anomaly 
Analytics with Watson that will do realtime/almost realtime analysis of the 
logs for you, and try to warn you about potential issues.

As others said, however, I don’t think using something like DFSORT will meet 
your needs, without a lot of work.  At the least I’d suggest using a 
pre-processing tool to handle the initial log parsing before you sort/summarize 
the data.
--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock


From: Jason Cai 
Date: Thursday, July 27, 2023 at 3:07 AM
To: IBM-MAIN@LISTSERV.UA.EDU , Kevin Mckenzie 

Cc: Jason Cai 
Subject: [EXTERNAL] Re: Inquiry about extracting and counting msgid from 
operlog using sort program
Dear Massimo and Kolusu,

Thank you for your help. I got the results I wanted. The results are as follows.
   MSGID NUMREC
  $HASP000   20
  $HASP001   10
  ..
>DISTINCT COUNT: 530

 I also appreciate Kevin McKenzie's reply.

 The reason why I counted how many times each unique msgid appeared in a day is 
to compare today's operlog with yesterday's operlog and

 see which msgids are new and which ones increased more than yesterday, such as 
by 30%.

 I got a list of msgids that need special attention. I used this list to 
extract the specific content of today's OPERLOG to find out the possible risks 
of the system in time.

I have two questions:

1.The first question is whether there is a better way or tool to analyze 
operlog.

2. The second question is whether sort can delete the numbers marked by line 
breaks in the multi-line records of MSGID to facilitate deduplication of 
multiple lines. See the information below:

•M 0090 HZS0002E CHECK 423

•E...   423  0090 IXCH0602E The sysplex couple data set has insufficient

The second line 423 indicates a line break of the first line 423.. I want to 
delete 423 of the first line. REXX processing is very easy but too slow because 
my operlog for one day is very large. Can sort or other tools handle it?

Any suggestions is greatly appreciated

Jason Cai



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Kevin Mckenzie]

Hi, Jason.  If all you’re looking to do is summarize log data, I can give you 
some options, at least.

There was/is a tool that Kevin Kelley, since retired, wrote to help clients do 
log analysis that would do exactly what you want, on platform, in batch; I’m 
trying to find it again.  It was available on testcase.boulder.ibm.com, and may 
well still be there in some subdirectory; I’ll spend a bit of time searching 
for it again.  Links to it have apparently been lost.  I believe it was 
mentioned in some Redbooks as well.  If I can find it, I’ll put it in the z/OS 
github repository.  It was intended to help clients do message suppression and 
the like.

I have a python library, zoslogs (https://pypi.org/project/zoslogs/), that I 
built to do very basic log parsing.  So if you wanted to, you could use it as a 
basis for something like what you’re trying to do; you at least wouldn’t have 
to worry about doing the initial parsing yourself.

There are some tools that might give you an easier interface to gather the 
syslog data, such as the z Common Data Provider, or the z/OSMF REST API.  I 
believe zoau does something similar.

There are also things like IBM Z Operational Log Analytics and IBM Z Anomaly 
Analytics with Watson that will do realtime/almost realtime analysis of the 
logs for you, and try to warn you about potential issues.

As others said, however, I don’t think using something like DFSORT will meet 
your needs, without a lot of work.  At the least I’d suggest using a 
pre-processing tool to handle the initial log parsing before you sort/summarize 
the data.
--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock


From: Jason Cai 
Date: Thursday, July 27, 2023 at 3:07 AM
To: IBM-MAIN@LISTSERV.UA.EDU , Kevin Mckenzie 

Cc: Jason Cai 
Subject: [EXTERNAL] Re: Inquiry about extracting and counting msgid from 
operlog using sort program
Dear Massimo and Kolusu,

Thank you for your help. I got the results I wanted. The results are as follows.
   MSGID NUMREC
  $HASP000   20
  $HASP001   10
  ..
>DISTINCT COUNT: 530

 I also appreciate Kevin McKenzie's reply.

 The reason why I counted how many times each unique msgid appeared in a day is 
to compare today's operlog with yesterday's operlog and

 see which msgids are new and which ones increased more than yesterday, such as 
by 30%.

 I got a list of msgids that need special attention. I used this list to 
extract the specific content of today's OPERLOG to find out the possible risks 
of the system in time.

I have two questions:

1.The first question is whether there is a better way or tool to analyze 
operlog.

2. The second question is whether sort can delete the numbers marked by line 
breaks in the multi-line records of MSGID to facilitate deduplication of 
multiple lines. See the information below:

•M 0090 HZS0002E CHECK 423

•E...   423  0090 IXCH0602E The sysplex couple data set has insufficient

The second line 423 indicates a line break of the first line 423.. I want to 
delete 423 of the first line. REXX processing is very easy but too slow because 
my operlog for one day is very large. Can sort or other tools handle it?

Any suggestions is greatly appreciated

Jason Cai



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Sri h Kolusu]

>> Can dfsort delete the last word if the first position is M and the last word 
>> is a number?

Jason,

Quite simple, you need to show me an example, the length of the last word and 
the DCB properties of the input file and I can show you how to do it.

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Massimo Biancucci]

Jason,

for you first question I could say experience.
All my customer sites have different "hot messages" to look at.
Others are only warning, others (the most of) are simply ignored.
I cannot simply share them.
I could give you some "generic hints" like:

   - JES Spool over x% then add a spool volume
   - IRA* messages that warn about critical memory issues
   - .

sorry for that.

About SORT, I can give you a sample to approximate what you need even
though it must be tailored to fit your big picture.
Assuming my input, SYSLOG (VBM), and no more than 28 words after the MSGID
and single word max 30 characters.
If some words are separated by more than one blank the multiple spaces will
be reduced to one.

I'm sure Sri can give you something better.

OPTION VLSCMP
 INCLUDE COND=(6,1,SS,EQ,C'M')
 INREC IFTHEN=(WHEN=INIT,
  PARSE=(%00=(ABSPOS=5,FIXLEN=59),
 %01=(ENDBEFR=C' ',FIXLEN=30,REPEAT=29)),
   BUILD=(1,4,%00,%01,%02,%03,%04,%05,%06,%07,%08,%09,
  %10,%11,%12,%13,%14,%15,%16,%17,%18,%19,
  %20,%21,%22,%23,%24,%25,%26,%27,%28,%29)),
IFTHEN=(WHEN=(904,3,CH,GE,C'000',AND,907,1,CH,EQ,C' '),
  OVERLAY=(904:30X)),
 IFTHEN=(WHEN=(874,3,CH,GE,C'000',AND,877,1,CH,EQ,C' '),
  OVERLAY=(874:30X)),
 IFTHEN=(WHEN=(844,3,CH,GE,C'000',AND,847,1,CH,EQ,C' '),
  OVERLAY=(844:30X)),
 IFTHEN=(WHEN=(814,3,CH,GE,C'000',AND,817,1,CH,EQ,C' '),
  OVERLAY=(814:30X)),
 IFTHEN=(WHEN=(784,3,CH,GE,C'000',AND,787,1,CH,EQ,C' '),
  OVERLAY=(784:30X)),
 IFTHEN=(WHEN=(754,3,CH,GE,C'000',AND,757,1,CH,EQ,C' '),
  OVERLAY=(754:30X)),
 IFTHEN=(WHEN=(724,3,CH,GE,C'000',AND,727,1,CH,EQ,C' '),
  OVERLAY=(724:30X)),
 IFTHEN=(WHEN=(694,3,CH,GE,C'000',AND,697,1,CH,EQ,C' '),
  OVERLAY=(694:30X)),
 IFTHEN=(WHEN=(664,3,CH,GE,C'000',AND,667,1,CH,EQ,C' '),
  OVERLAY=(664:30X)),
 IFTHEN=(WHEN=(634,3,CH,GE,C'000',AND,637,1,CH,EQ,C' '),
  OVERLAY=(634:30X)),
 IFTHEN=(WHEN=(604,3,CH,GE,C'000',AND,607,1,CH,EQ,C' '),
  OVERLAY=(604:30X)),
 IFTHEN=(WHEN=(574,3,CH,GE,C'000',AND,577,1,CH,EQ,C' '),
  OVERLAY=(574:30X)),
 IFTHEN=(WHEN=(544,3,CH,GE,C'000',AND,547,1,CH,EQ,C' '),
  OVERLAY=(544:30X)),
 IFTHEN=(WHEN=(514,3,CH,GE,C'000',AND,517,1,CH,EQ,C' '),
  OVERLAY=(514:30X)),
 IFTHEN=(WHEN=(484,3,CH,GE,C'000',AND,487,1,CH,EQ,C' '),
  OVERLAY=(484:30X)),
 IFTHEN=(WHEN=(454,3,CH,GE,C'000',AND,457,1,CH,EQ,C' '),
  OVERLAY=(454:30X)),
 IFTHEN=(WHEN=(424,3,CH,GE,C'000',AND,427,1,CH,EQ,C' '),
  OVERLAY=(424:30X)),
 IFTHEN=(WHEN=(394,3,CH,GE,C'000',AND,397,1,CH,EQ,C' '),
  OVERLAY=(394:30X)),
 IFTHEN=(WHEN=(364,3,CH,GE,C'000',AND,367,1,CH,EQ,C' '),
  OVERLAY=(364:30X)),
 IFTHEN=(WHEN=(334,3,CH,GE,C'000',AND,337,1,CH,EQ,C' '),
  OVERLAY=(334:30X)),
 IFTHEN=(WHEN=(304,3,CH,GE,C'000',AND,307,1,CH,EQ,C' '),
  OVERLAY=(304:30X)),
 IFTHEN=(WHEN=(274,3,CH,GE,C'000',AND,277,1,CH,EQ,C' '),
  OVERLAY=(274:30X)),
 IFTHEN=(WHEN=(244,3,CH,GE,C'000',AND,247,1,CH,EQ,C' '),
  OVERLAY=(244:30X)),
 IFTHEN=(WHEN=(214,3,CH,GE,C'000',AND,217,1,CH,EQ,C' '),
  OVERLAY=(214:30X)),
 IFTHEN=(WHEN=(184,3,CH,GE,C'000',AND,187,1,CH,EQ,C' '),
  OVERLAY=(184:30X)),
 IFTHEN=(WHEN=(154,3,CH,GE,C'000',AND,157,1,CH,EQ,C' '),
  OVERLAY=(154:30X)),
 IFTHEN=(WHEN=(124,3,CH,GE,C'000',AND,127,1,CH,EQ,C' '),
  OVERLAY=(124:30X)),
 IFTHEN=(WHEN=(094,3,CH,GE,C'000',AND,097,1,CH,EQ,C' '),
  OVERLAY=(094:30X)),
 IFTHEN=(WHEN=(064,3,CH,GE,C'000',AND,067,1,CH,EQ,C' '),
  OVERLAY=(064:30X))
 SORT FIELDS=COPY
 OUTFIL FNAMES=SORTOUT,
  OUTREC=(1,4,5,59,
  64,870,SQZ=(SHIFT=LEFT,MID=C' '))

Best regards.
Max

Il giorno gio 27 lug 2023 alle ore 14:25 Jason Cai  ha
scritto:

> Dear Max
>
> I am writing to you regarding two questions that I have.
>
> The first question is about how to monitor and alert the system issues. We
> can monitor most of the issues and send alerts to our phones. However, when
> we cannot provide a complete list of msgids to the monitoring system, some
> critical alerts may be missed. How can we minimize the missing of critical
> alerts and also reduce the false alarms? This is our biggest challenge. We
> want to analyze the operlog every day to find out if there are any
> potential risks in the system and also to locate the problem when it
> occurs. I wonder how your shop handles this situation.
>
> The second question is a simplified one. Can dfsort delete the last word
> if the first position is M and the last word is a number?
>
> I appreciate your time and attention.
>
> Sincerely,
> Jason Cai
>
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Jason Cai]

Dear Max

I am writing to you regarding two questions that I have.

The first question is about how to monitor and alert the system issues. We can 
monitor most of the issues and send alerts to our phones. However, when we 
cannot provide a complete list of msgids to the monitoring system, some 
critical alerts may be missed. How can we minimize the missing of critical 
alerts and also reduce the false alarms? This is our biggest challenge. We want 
to analyze the operlog every day to find out if there are any potential risks 
in the system and also to locate the problem when it occurs. I wonder how your 
shop handles this situation.

The second question is a simplified one. Can dfsort delete the last word if the 
first position is M and the last word is a number?

I appreciate your time and attention.

Sincerely,
Jason Cai

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Massimo Biancucci]

Jason,

as usually, it depends.

About Operlog analysis, it depends on the response-time you need.
At different customer sites I use automation product (BMC, IBM, Broadcom)
in order to catch specific messages and react asap with specific actions or
simply sending SMS/Mail.
Some others use external product (Splunk and similar) to collect, correlate
messages, showing nice graphic screens and sometimes react.

About the second question, I don't think there's a simply solutions with
SORT because the text of the message is really variable, the word count is
message specific and finally we cannot be sure that a "number" as last word
is a continuation hint.
Could you please better explain your goal ?

Best regards.
Max

Il giorno gio 27 lug 2023 alle ore 09:08 Jason Cai  ha
scritto:

> Dear Massimo and Kolusu,
>
> Thank you for your help. I got the results I wanted. The results are as
> follows.
>MSGID NUMREC
>   $HASP000   20
>   $HASP001   10
>   ..
> >DISTINCT COUNT: 530
>
>  I also appreciate Kevin McKenzie's reply.
>
>  The reason why I counted how many times each unique msgid appeared in a
> day is to compare today's operlog with yesterday's operlog and
>
>  see which msgids are new and which ones increased more than yesterday,
> such as by 30%.
>
>  I got a list of msgids that need special attention. I used this list to
> extract the specific content of today's OPERLOG to find out the possible
> risks of the system in time.
>
> I have two questions:
>
> 1.The first question is whether there is a better way or tool to analyze
> operlog.
>
> 2. The second question is whether sort can delete the numbers marked by
> line breaks in the multi-line records of MSGID to facilitate deduplication
> of multiple lines. See the information below:
>
> •M 0090 HZS0002E CHECK 423
>
> •E...   423  0090 IXCH0602E The sysplex couple data set has
> insufficient
>
> The second line 423 indicates a line break of the first line 423.. I want
> to delete 423 of the first line. REXX processing is very easy but too slow
> because my operlog for one day is very large. Can sort or other tools
> handle it?
>
> Any suggestions is greatly appreciated
>
> Jason Cai
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Jason Cai]

Dear Massimo and Kolusu,

Thank you for your help. I got the results I wanted. The results are as follows.
   MSGID NUMREC 
  $HASP000   20 
  $HASP001   10
  ..
>DISTINCT COUNT: 530

 I also appreciate Kevin McKenzie's reply.

 The reason why I counted how many times each unique msgid appeared in a day is 
to compare today's operlog with yesterday's operlog and

 see which msgids are new and which ones increased more than yesterday, such as 
by 30%.

 I got a list of msgids that need special attention. I used this list to 
extract the specific content of today's OPERLOG to find out the possible risks 
of the system in time.

I have two questions:

1.The first question is whether there is a better way or tool to analyze 
operlog.

2. The second question is whether sort can delete the numbers marked by line 
breaks in the multi-line records of MSGID to facilitate deduplication of 
multiple lines. See the information below:

•M 0090 HZS0002E CHECK 423

•E...   423  0090 IXCH0602E The sysplex couple data set has insufficient

The second line 423 indicates a line break of the first line 423.. I want to 
delete 423 of the first line. REXX processing is very easy but too slow because 
my operlog for one day is very large. Can sort or other tools handle it?  

Any suggestions is greatly appreciated 

Jason Cai

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Sri h Kolusu]

≫ INREC PARSE=(%00=(ABSPOS=64,ENDBEFR=C' ',FIXLEN=15)),

Massimo,

Jason wants to count the MSGID , not the message TEXT which starts at position 
58, but your parse is starting with ABSPOS=64.


Thanks,
Kolusu



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Massimo Biancucci]

Jason,

you can try this:

OPTION VLSCMP
INCLUDE COND=(6,1,SS,EQ,C'NXWM',AND,
 7,1,CH,EQ,C' ',AND,
 64,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')
INREC PARSE=(%00=(ABSPOS=64,ENDBEFR=C' ',FIXLEN=15)),
 BUILD=(1,4,%00,X'0001')
SORT FIELDS=(5,15,BI,A)
SUM FIELDS=(20,4,BI)
OUTFIL FNAMES=SORTOUT,VTOF,
 HEADER1=('MSGID NUMREC'),
 OUTREC=(5,15,X,20,4,BI,EDIT=(IIIT)),
 TRAILER1=('>DISTINCT COUNT:',COUNT)

My syslog dataset is VBM so modify accordingly to your definition.

It assumes a max length for message of 15, starting with a capital letter
and a blank as separator from the message text itself.

>From the manual:

"

Each SYSLOG record is prefaced by a two-character record type field.

Valid first characters are:

   - N - single-line message
   - W - single-line message with reply
  - WTOR messages are not processed by Message Flood Automation.
   - M - first line of a multi-line message
  - Message Flood Automation can only react to the first line of a
  multiline message, not to any of the label, data or end lines
   - L - multi-line message label line
   - D - multi-line message data line
   - E - multi-line message data/end line
   - S - continuation of previous line
   - O - LOG command input
   - X - non-hardcopy or LOG command source

Valid second characters are:

   - C - command issued by operator
   - R - command response message
   - I - internally issued command
   - U - command from unknown console ID (z/OS® R8 and above)

"

Best regards.

Max



Il giorno mer 26 lug 2023 alle ore 10:19 Jason Cai  ha
scritto:

> Dear All,
>
>  I am writing to inquire about how to use the sort program on the
> mainframe to extract and count msgid from a day's operlog. I would
> appreciate your guidance and advice on this matter.
>
> Msgid is a keyword that identifies system messages. It usually consists of
> 7 letters and a number, such as IEA001I or CSV003E. However, some msgid may
> have up to 10 characters.
>
> I need to extract all msgid from the operlog, and remove any duplicates.
> Then, I need to count the number of occurrences of each msgid and generate
> a report.
>
> I have tried to use the following statement to extract msgid, and it seems
> to work well. However, I am not sure if there is anything missing or if it
> can be written more simply.
>
> SORT FIELDS=(58,7,CH,A)
>  INCLUDE COND=((58,7,SS,NE,C' ',AND,
>65,1,CH,EQ,C' ',AND,
>(63,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
>(64,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
>(58,8,SS,NE,C' ',AND,
>66,1,CH,EQ,C' ',AND,
>(64,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
>(65,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
>(58,9,SS,NE,C' ',AND,
> 67,1,CH,EQ,C' ',AND,
>(65,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
>(66,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
>(58,10,SS,NE,C' ',AND,
> 68,1,CH,EQ,C' ',AND,
>(66,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
>(67,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ'))
>
> But I do not know how to count the number of occurrences of each msgid and
> eliminate duplicates.
>
> Thank you for your time and attention.
>
> Sincerely,
>
> Jason Cai
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Sri h Kolusu]

>> But I do not know how to count the number of occurrences of each msgid and 
>> eliminate duplicates.

Jason,

Assuming the msgid is at position 58 for a length of 7 bytes, you can use the 
following control cards that would give you the desired results.

//SYSINDD *
  INCLUDE COND=((58,7,SS,NE,C' ',AND,
 65,1,CH,EQ,C' ',AND,
(63,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
(64,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
(58,8,SS,NE,C' ',AND,
 66,1,CH,EQ,C' ',AND,
(64,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
(65,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
(58,9,SS,NE,C' ',AND,
 67,1,CH,EQ,C' ',AND,
(65,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
(66,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
(58,10,SS,NE,C' ',AND,
 68,1,CH,EQ,C' ',AND,
(66,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
(67,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')))

  SORT FIELDS=(58,7,CH,A)
  SUM FIELDS=NONE
  OUTFIL REMOVECC,
  TRAILER1=('COUNT OF UNIQUE MSGID IS :',COUNT)
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Inquiry about extracting and counting msgid from operlog using sort program

[Kevin Mckenzie]

What are you actually trying to do with this information?  There are various 
tools available for log analysis.

--
Kevin McKenzie

External Phone: 845-435-8282, Tie-line: 8-295-8282
z/OS Test Services - Test Architect, Provisioning
z/OS Hardware/Software Interlock


From: IBM Mainframe Discussion List  on behalf of 
Jason Cai 
Date: Wednesday, July 26, 2023 at 4:19 AM
To: IBM-MAIN@LISTSERV.UA.EDU 
Subject: [EXTERNAL] Inquiry about extracting and counting msgid from operlog 
using sort program
Dear All,

 I am writing to inquire about how to use the sort program on the mainframe to 
extract and count msgid from a day's operlog. I would appreciate your guidance 
and advice on this matter.

Msgid is a keyword that identifies system messages. It usually consists of 7 
letters and a number, such as IEA001I or CSV003E. However, some msgid may have 
up to 10 characters.

I need to extract all msgid from the operlog, and remove any duplicates. Then, 
I need to count the number of occurrences of each msgid and generate a report.

I have tried to use the following statement to extract msgid, and it seems to 
work well. However, I am not sure if there is anything missing or if it can be 
written more simply.

SORT FIELDS=(58,7,CH,A)
 INCLUDE COND=((58,7,SS,NE,C' ',AND,
   65,1,CH,EQ,C' ',AND,
   (63,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
   (64,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,8,SS,NE,C' ',AND,
   66,1,CH,EQ,C' ',AND,
   (64,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
   (65,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,9,SS,NE,C' ',AND,
67,1,CH,EQ,C' ',AND,
   (65,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
   (66,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,10,SS,NE,C' ',AND,
68,1,CH,EQ,C' ',AND,
   (66,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
   (67,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ'))

But I do not know how to count the number of occurrences of each msgid and 
eliminate duplicates.

Thank you for your time and attention.

Sincerely,

Jason Cai

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Inquiry about extracting and counting msgid from operlog using sort program

[Jason Cai]

Dear All,

 I am writing to inquire about how to use the sort program on the mainframe to 
extract and count msgid from a day's operlog. I would appreciate your guidance 
and advice on this matter.

Msgid is a keyword that identifies system messages. It usually consists of 7 
letters and a number, such as IEA001I or CSV003E. However, some msgid may have 
up to 10 characters.

I need to extract all msgid from the operlog, and remove any duplicates. Then, 
I need to count the number of occurrences of each msgid and generate a report.

I have tried to use the following statement to extract msgid, and it seems to 
work well. However, I am not sure if there is anything missing or if it can be 
written more simply.

SORT FIELDS=(58,7,CH,A)   
 INCLUDE COND=((58,7,SS,NE,C' ',AND,  
   65,1,CH,EQ,C' ',AND,   
   (63,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,   
   (64,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,8,SS,NE,C' ',AND,  
   66,1,CH,EQ,C' ',AND,   
   (64,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,   
   (65,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,9,SS,NE,C' ',AND,  
67,1,CH,EQ,C' ',AND,  
   (65,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,   
   (66,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ')),OR,
   (58,10,SS,NE,C' ',AND, 
68,1,CH,EQ,C' ',AND,  
   (66,1,SS,EQ,C'0,1,2,3,4,5,6,7,8,9'),AND,
   (67,1,SS,EQ,C'ABCDEFGHIJKLMNOPQRSTUVWXYZ'))

But I do not know how to count the number of occurrences of each msgid and 
eliminate duplicates.

Thank you for your time and attention.

Sincerely,

Jason Cai

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort INCLUDE problem

[Sri h Kolusu]

>> You job is not summarizing by HLQ; instead is creating one record by each 
>> input record with HLQ and size.

Jack,

Now that I know what you are trying to do, it is quite easy to add the summary 
by HLQ in the same step.  You have 2 different multiplications so we need to 
use IFTHEN statements

So, please change the SORT001 SYSIN to the following (Untested).

//SYSINDD *,SYMBOLS=JCLONLY
  INREC IFTHEN=(WHEN=INIT,
 PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
   OVERLAY=(BRS-HLQ:%01)),

IFTHEN=(WHEN=(INP-MGMTCL,EQ,C''),
 OVERLAY=(BRS-BYT:INP-FLSIZE,MUL,+4,PD,LENGTH=16)),

IFTHEN=(WHEN=NONE,
 OVERLAY=(BRS-BYT:INP-FLSIZE,MUL,+2,PD,LENGTH=16))

  SORT FIELDS=(BRS-HLQ,A,
   INP-MGMTCL,A)

  OUTFIL FNAMES=HLQ#BRS,
  INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
   INP-MGMTCL,EQ,C''),
  REMOVECC,NODETAIL,
  HEADER1=(01:'HLQ   BRS TOTAL KB',/,
   01:' - ---'),
  SECTIONS=(BRS-HLQ,
  TRAILER3=(X,
BRS-HLQ,
X,
TOT=(BRS-BYT,EDIT=(I.III.III.III.III.IIT

  OUTFIL FNAMES=HLQ#LX,
  INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
   INP-MGMTCL,NE,C''),
  REMOVECC,NODETAIL,
  HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
   01:' - ---'),
  SECTIONS=(BRS-HLQ,
  TRAILER3=(X,
BRS-HLQ,
X,
TOT=(BRS-BYT,EDIT=(I.III.III.III.III.IIT

/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort INCLUDE problem

[Jack Zukt]

Hello Kolusu,

I think that this DFSORT statements should work:

SKIP,1
BRS-BYT,*,16,PD
LX-BYT,*,16,PD
//SYSINDD *,SYMBOLS=EXECSYS
  INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
OVERLAY=(BRS-HLQ:%01,
 BRS-BYT:INP-FLSIZE,TO=PD,LENGTH=16,
 LX-BYT:INP-FLSIZE,TO=PD,LENGTH=16)
   SUM FIELDS=(BRS-BYT,LX-BYT)
  SORT FIELDS=(BRS-HLQ,A)
 OUTFIL FNAMES=HLQ#BRS,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,EQ,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
BRS-HLQ,
X,
BRS-BYT,MUL,+4,LENGTH=20,
EDIT=(I.III.III.III.III.IIT))
 OUTFIL FNAMES=HLQ#LX,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,NE,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
BRS-HLQ,
X,
LX-BYT,MUL,+2,LENGTH=20,
EDIT=(I.III.III.III.III.IIT))
/*

However I do not understand why there is at least one HLQ that should be
present on both files is present only in one, the one referenced by the
first OUTFIL.
Best wishes
Jack

On Fri, 28 Apr 2023 at 16:20, Jack Zukt  wrote:

> Hello Kolusu,
>
> Thank you for your quick answer.
> As you have guessed, my problem is on the two first steps, which you have
> consolidated in one. The management class is only relevant for spiting the
> information into two files; what I need is to summarize the size by high
> level qualifier for the files with that specific management class, and for
> the files with any other management class. The tapes that have that
> management class have four copies (hence the MUL,+4) the tapes that have
> other management classes have only two copies (so, MUL,+2).
> The problem is that at least one high level qualifier should be present on
> both files but is only showing up in one. I wonder if that is the only one.
> You job is not summarizing by HLQ; instead is creating one record by each
> input record with HLQ and size.
> Best wishes
> Jack
>
> On Fri, 28 Apr 2023 at 15:44, Sri h Kolusu  wrote:
>
>> >> What is troubling me is that I have at least one HLQ that is present
>> on both  files with the records separated by the management class, but that
>> HLQ is only present on the summarized by HLQ file that does not have that
>> specific management class. And the INCLUDE that I am using is the same for
>> both cases
>>
>> Jack,
>>
>> It is quite difficult to debug the issue without knowing which step is
>> causing the error.  You have 3 different steps and all of them are using
>> the INCLUDE cond.  But if I had to take an educated guess it is because the
>> first 2 steps are summarizing, and you will not have the MGMT class when
>> doing that as you are ONLY sorting on the HLQ.  Technically you don't need
>> SORT002 step as you can do it in SORT001 step itself.
>>
>> So, try this JCL (combined 001 and 002 steps into a single one)
>>
>> //SORT001  EXEC PGM=SORT
>> //SYSOUT   DD  SYSOUT=*
>> //SYMNOUT  DD  SYSOUT=*
>> //SORTIN   DD  DISP=SHR,DSN=
>> //HLQ#BRS  DD  DSN=,
>> // DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
>> // RECFM=FB,LRECL=30
>> //HLQ#LX   DD  DSN=,
>> // DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
>> // RECFM=FB,LRECL=30
>> //SYMNAMES DD *
>> INP-DSNAME,02,44,CH
>> INP-CRDATE,47,10,CH
>> INP-LRDATE,58,10,CH
>> INP-XPDATE,69,10,CH
>> INP-FLSIZE,80,10,UFF
>> INP-K#CHAR,90,01,CH
>> INP-MGMTCL,92,30,CH
>> SKIP,1
>> BRS-HLQ,*,08,CH
>> SKIP,1
>> BRS-BYT,*,16,PD
>> //SYSINDD *,SYMBOLS=EXECSYS
>>   INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
>> OVERLAY=(BRS-HLQ:%01,
>>  BRS-BYT:INP-FLSIZE,MUL,+4,PD,LENGTH=16)
>>
>>   SORT FIELDS=(BRS-HLQ,A,
>>INP-MGMTCL,A)
>>
>>  OUTFIL FNAMES=HLQ#BRS,
>>INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
>> INP-MGMTCL,EQ,C''),
>>REMOVECC,
>>HEADER1=(01:'HLQ   BRS TOTAL KB',/,
>> 01:' - ---'),
>>  BUILD=(X,
>> BRS-HLQ,
>> X,
>> BRS-BYT,EDIT=(I.III.III.III.III.IIT))
>>
>>  OUTFIL FNAMES=HLQ#LX,
>>INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
>> INP-MGMTCL,NE,C''),
>>REMOVECC,
>>HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
>> 

Re: Sort INCLUDE problem

[Jack Zukt]

Hello Kolusu,

Thank you for your quick answer.
As you have guessed, my problem is on the two first steps, which you have
consolidated in one. The management class is only relevant for spiting the
information into two files; what I need is to summarize the size by high
level qualifier for the files with that specific management class, and for
the files with any other management class. The tapes that have that
management class have four copies (hence the MUL,+4) the tapes that have
other management classes have only two copies (so, MUL,+2).
The problem is that at least one high level qualifier should be present on
both files but is only showing up in one. I wonder if that is the only one.
You job is not summarizing by HLQ; instead is creating one record by each
input record with HLQ and size.
Best wishes
Jack

On Fri, 28 Apr 2023 at 15:44, Sri h Kolusu  wrote:

> >> What is troubling me is that I have at least one HLQ that is present on
> both  files with the records separated by the management class, but that
> HLQ is only present on the summarized by HLQ file that does not have that
> specific management class. And the INCLUDE that I am using is the same for
> both cases
>
> Jack,
>
> It is quite difficult to debug the issue without knowing which step is
> causing the error.  You have 3 different steps and all of them are using
> the INCLUDE cond.  But if I had to take an educated guess it is because the
> first 2 steps are summarizing, and you will not have the MGMT class when
> doing that as you are ONLY sorting on the HLQ.  Technically you don't need
> SORT002 step as you can do it in SORT001 step itself.
>
> So, try this JCL (combined 001 and 002 steps into a single one)
>
> //SORT001  EXEC PGM=SORT
> //SYSOUT   DD  SYSOUT=*
> //SYMNOUT  DD  SYSOUT=*
> //SORTIN   DD  DISP=SHR,DSN=
> //HLQ#BRS  DD  DSN=,
> // DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
> // RECFM=FB,LRECL=30
> //HLQ#LX   DD  DSN=,
> // DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
> // RECFM=FB,LRECL=30
> //SYMNAMES DD *
> INP-DSNAME,02,44,CH
> INP-CRDATE,47,10,CH
> INP-LRDATE,58,10,CH
> INP-XPDATE,69,10,CH
> INP-FLSIZE,80,10,UFF
> INP-K#CHAR,90,01,CH
> INP-MGMTCL,92,30,CH
> SKIP,1
> BRS-HLQ,*,08,CH
> SKIP,1
> BRS-BYT,*,16,PD
> //SYSINDD *,SYMBOLS=EXECSYS
>   INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
> OVERLAY=(BRS-HLQ:%01,
>  BRS-BYT:INP-FLSIZE,MUL,+4,PD,LENGTH=16)
>
>   SORT FIELDS=(BRS-HLQ,A,
>INP-MGMTCL,A)
>
>  OUTFIL FNAMES=HLQ#BRS,
>INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
> INP-MGMTCL,EQ,C''),
>REMOVECC,
>HEADER1=(01:'HLQ   BRS TOTAL KB',/,
> 01:' - ---'),
>  BUILD=(X,
> BRS-HLQ,
> X,
> BRS-BYT,EDIT=(I.III.III.III.III.IIT))
>
>  OUTFIL FNAMES=HLQ#LX,
>INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
> INP-MGMTCL,NE,C''),
>REMOVECC,
>HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
> 01:' - ---'),
>  BUILD=(X,
> BRS-HLQ,
> X,
> BRS-BYT,EDIT=(I.III.III.III.III.IIT))
> /*
>
>
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sort INCLUDE problem

[Sri h Kolusu]

>> What is troubling me is that I have at least one HLQ that is present on both 
>>  files with the records separated by the management class, but that HLQ is 
>> only present on the summarized by HLQ file that does not have that specific 
>> management class. And the INCLUDE that I am using is the same for both cases

Jack,

It is quite difficult to debug the issue without knowing which step is causing 
the error.  You have 3 different steps and all of them are using the INCLUDE 
cond.  But if I had to take an educated guess it is because the first 2 steps 
are summarizing, and you will not have the MGMT class when doing that as you 
are ONLY sorting on the HLQ.  Technically you don't need SORT002 step as you 
can do it in SORT001 step itself.

So, try this JCL (combined 001 and 002 steps into a single one)

//SORT001  EXEC PGM=SORT
//SYSOUT   DD  SYSOUT=*
//SYMNOUT  DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=
//HLQ#BRS  DD  DSN=,
// DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
// RECFM=FB,LRECL=30
//HLQ#LX   DD  DSN=,
// DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
// RECFM=FB,LRECL=30
//SYMNAMES DD *
INP-DSNAME,02,44,CH
INP-CRDATE,47,10,CH
INP-LRDATE,58,10,CH
INP-XPDATE,69,10,CH
INP-FLSIZE,80,10,UFF
INP-K#CHAR,90,01,CH
INP-MGMTCL,92,30,CH
SKIP,1
BRS-HLQ,*,08,CH
SKIP,1
BRS-BYT,*,16,PD
//SYSINDD *,SYMBOLS=EXECSYS
  INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
OVERLAY=(BRS-HLQ:%01,
 BRS-BYT:INP-FLSIZE,MUL,+4,PD,LENGTH=16)

  SORT FIELDS=(BRS-HLQ,A,
   INP-MGMTCL,A)

 OUTFIL FNAMES=HLQ#BRS,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,EQ,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
BRS-HLQ,
X,
BRS-BYT,EDIT=(I.III.III.III.III.IIT))

 OUTFIL FNAMES=HLQ#LX,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,NE,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
BRS-HLQ,
X,
BRS-BYT,EDIT=(I.III.III.III.III.IIT))
/*



Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Sort INCLUDE problem

[Jack Zukt]

Hi
I have this file in which each record has a tape dataset name, the creation
date, the last ref date, size and management class.
I need to summarize the size by high level qualifier and to create a file
with all the record that have one specific management class, and do the
same for the records that have a different management class.
What is troubling me is that I have at least one HLQ that is present on
both  files with the records separated by the management class, but that
HLQ is only present on the summarized by HLQ file that does not have that
specific management class. And the INCLUDE that I am using is the same for
both cases

INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,EQ,C''),

INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,NE,C''),

So, what am I missing?

This is the JCL that I am using:

//*
//EXP#VARS EXPORT SYMLIST=MGMT
//*
//SET#IHLQ SET IHLQ=INHLQ
//SET#OHLQ SET OHLQ=OUTHLQ
//SET#MGMT SET MGMT=MCVT2DR
//*
//APAGA001 EXEC PGM=IEFBR14
//DL001DD  DSN=,
// DISP=(MOD,DELETE),SPACE=(TRK,(1))
//DL002DD  DSN=,
// DISP=(MOD,DELETE),SPACE=(TRK,(1))
//DL003DD  DSN=,
// DISP=(MOD,DELETE),SPACE=(TRK,(1))
//DL004DD  DSN=,
// DISP=(MOD,DELETE),SPACE=(TRK,(1))
//*
//* +-+
//* | |
//* |  SUMMARIZE THE BRS USED TAPE SPACE BY HIGH LEVEL QUALIFIER  |
//* | |
//* +-+
//SORT001  EXEC PGM=SORT
//SYSOUT   DD  SYSOUT=*
//SYMNOUT  DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=
//HLQ#BRS  DD  DSN=,
// DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
// RECFM=FB,LRECL=30
//SYMNAMES DD *
INP-DSNAME,02,44,CH
INP-CRDATE,47,10,CH
INP-LRDATE,58,10,CH
INP-XPDATE,69,10,CH
INP-FLSIZE,80,10,UFF
INP-K#CHAR,90,01,CH
INP-MGMTCL,92,30,CH
SKIP,1
BRS-HLQ,*,08,CH
SKIP,1
BRS-BYT,*,16,PD
//SYSINDD *,SYMBOLS=EXECSYS

  INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
OVERLAY=(BRS-HLQ:%01,
 BRS-BYT:INP-FLSIZE,TO=PD,LENGTH=16)
  SORT FIELDS=(BRS-HLQ,A)
   SUM FIELDS=(BRS-BYT)
 OUTFIL FNAMES=HLQ#BRS,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,EQ,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
BRS-HLQ,
X,
BRS-BYT,MUL,+4,LENGTH=20,
EDIT=(I.III.III.III.III.IIT))
/*
//*
//* +---+
//* |   |
//* | SUMMARIZE THE NOT BRS USED TAPE SPACE BY HIGH LEVEL QUALIFIER |
//* |   |
//* +---+
//SORT002  EXEC PGM=SORT
//SYSOUT   DD  SYSOUT=*
//SYMNOUT  DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=
//HLQ#LX   DD  DSN=,
// DISP=(NEW,CATLG),SPACE=(TRK,(1,1),RLSE),
// RECFM=FB,LRECL=30
//SYMNAMES DD *
INP-DSNAME,02,44,CH
INP-CRDATE,47,10,CH
INP-LRDATE,58,10,CH
INP-XPDATE,69,10,CH
INP-FLSIZE,80,10,UFF
INP-K#CHAR,90,01,CH
INP-MGMTCL,92,30,CH
SKIP,1
LX-HLQ,*,08,CH
SKIP,1
LX-BYT,*,16,PD
//SYSINDD *,SYMBOLS=EXECSYS
  INREC PARSE=(%01=(ABSPOS=2,ENDBEFR=C'.',FIXLEN=8)),
OVERLAY=(LX-HLQ:%01,
 LX-BYT:INP-FLSIZE,TO=PD,LENGTH=16)
  SORT FIELDS=(LX-HLQ,A)
   SUM FIELDS=(LX-BYT)
 OUTFIL FNAMES=HLQ#LX,
   INCLUDE=(INP-K#CHAR,EQ,C'K',AND,
INP-MGMTCL,NE,C''),
   REMOVECC,
   HEADER1=(01:'HLQ   NON BRS TOTAL KB',/,
01:' - ---'),
 BUILD=(X,
LX-HLQ,
X,
LX-BYT,MUL,+2,LENGTH=20,
EDIT=(I.III.III.III.III.IIT))
/*
//*
//* +---+
//* |   |
//* |LIST THE BRS TAPE FILES|
//* |LIST THE NOT BRS TAPE FILES|
//* |   |
//* +---+
//SORT003  EXEC PGM=SORT
//SYSOUT   DD  SYSOUT=*
//SYMNOUT  DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=
//BRS#RMM DD   DSN=,
// DISP=(NEW,CATLG),SPACE=(CYL,(150,150),RLSE),
// RECFM=FB,LRECL=87
//LX#RMM DDDSN=,
// DISP=(NEW,CATLG),SPACE=(CYL,(150,150),RLSE),
// RECFM=FB,LRECL=87
//SYMNAMES DD *
INP-DSNAME,02,44,CH
INP-CRDATE,47,10,CH
INP-LRDATE,58,10,CH
INP-XPDATE,69,10,CH
INP-FLSIZE,80,10,UFF
INP-K#CHAR,90,01,CH
INP-MGMTCL,92,30,CH
//SYSIN DD *,SYMBOLS=EXECSYS
  OPTION VLSCMP,DYNALLOC=(,4)
  SORT FIELDS=(INP

Re: SORT SUM Problem

[Jack Zukt]

Thank you Sri.
It was as simple as that.
Many thanks
Jack

On Tue, 31 Jan 2023 at 12:46, Sri h Kolusu  wrote:

> Jack,
>
> The issue is you have NOT converted the UFF value to Binary before
> performing the SUM
>
> Here is the change you need (Untested)
>
> //SYSINDD *
>   INREC PARSE=(%01=(ABSPOS=1,ENDBEFR=C'.',FIXLEN=8)),
> BUILD=(FMT-HLQ:%01,
>FMT-BYTE:INP-BYTE,TO=BI,LENGTH=8)
>
>   SORT FIELDS=(FMT-HLQ,A)
>SUM FIELDS=(FMT-BYTE)
>   OUTREC BUILD=(X,
> FMT-HLQ,
> X,
> FMT-BYTE,TO=ZD,LENGTH=16)
> /*
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SORT SUM Problem

[Sri h Kolusu]

Jack,

The issue is you have NOT converted the UFF value to Binary before performing 
the SUM

Here is the change you need (Untested)

//SYSINDD *
  INREC PARSE=(%01=(ABSPOS=1,ENDBEFR=C'.',FIXLEN=8)),
BUILD=(FMT-HLQ:%01,
   FMT-BYTE:INP-BYTE,TO=BI,LENGTH=8)

  SORT FIELDS=(FMT-HLQ,A)
   SUM FIELDS=(FMT-BYTE)
  OUTREC BUILD=(X,
FMT-HLQ,
X,
FMT-BYTE,TO=ZD,LENGTH=16)
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SORT SUM Problem

[Jack Zukt]

Hi all,
I should probably say "my problem with SORT SUM", as the real problem is
not with SORT.

So, I have an input file with a 100 byte record that has a dataset name on
the first 44 bytes and a numeric value (UFF) on the last ten.
I need to sum up those 10 last bytes by High Level Qualifier.
My problem, after a few variations, is that I cannot get rid of the

ICE152I 0 OVERFLOW DURING SUMMATION - RC=0

message.

This is my latest try:

 //SYMNAMES DD *
 INP-DSN,01,44,CH
 INP-BYTE,91,10,UFF
 SKIP,1
 FMT-HLQ,1,08,CH
 FMT-BYTE,*,08,BI
 TMP-BYT1,=,04,BI
 TMP-BYT2,*,04,BI

//SYSINDD *
  INREC PARSE=(%01=(ABSPOS=1,ENDBEFR=C'.',FIXLEN=8)),
OVERLAY=(FMT-HLQ:%01,
 TMP-BYT2:INP-BYTE) * SUM HLQ ALLOC SPACE
  SORT FIELDS=(FMT-HLQ,A)
   SUM FIELDS=(FMT-BYTE)
  OUTREC BUILD=(X,
FMT-HLQ,
X,
FMT-BYTE,TO=ZD,LENGTH=16)
/*

The SORTOUT is deirected to SYSOUT and it looks something like this:

COMUM9242510529872113
COMUM9242510529872114
COMUM9242510529872114
COMUM9242510529872114
COMUM9242510529871936
COMUM9242510529871936
COMUM9242510529872113
COMUM9242510529872113
COMUM9242510529872113

There are 26 of those when there should be only one (not counting the few
hundreds more repetitions of several HLQs)
So, what am I doing wrong?

Your help will be, as usual, greatly appreciated,
Jack

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to SORT Variable records with OVERLAY

[Billy Ashton]

Hi Kolusu...I have to toss in the white flag here, as I tried a few 
different things unsuccessfully. I have attached a groomed file of the 
input (it is the output of an IDCAMS Listcat for certain sales report 
files), and below is a second file for my second part of this. Here is 
what I am trying to get:
1. From the groomed File1 - a VBA 133 file of LISTCAT data - I am trying 
to get a list of the files and their created date, sorted in order of 
division, order number, and descending document creation date.
2. Then, I am trying to match that against the second file which has the 
latest processed document for each order to show two different reports:
A. Any document that has not been processed for an order (1 to n records 
each), and

B. The most recent document that has not been processed (1 record only)

Here is my SORT for #1, which works great to copy, but does not work to 
sort:

//SYSINDD *
  OPTION VLSCMP,VLSHRT
  SORT FIELDS=COPY
* SORT FIELDS=(116,08,CH,A,124,03,CH,A,135,7,CH,D)
*
  INCLUDE COND=(06,07,CH,EQ,C'NONVSAM',OR,
42,10,CH,EQ,C'CREATION--')
*
  INREC  IFOUTLEN=150,IFTHEN=(WHEN=GROUP,
BEGIN=(06,07,CH,EQ,C'NONVSAM'),
 PUSH=(69:22,44)),
 IFTHEN=(WHEN=(42,10,CH,EQ,C'CREATION--'),
  PARSE=(%=(ENDBEFR=C'.',ABSPOS=65),
   %=(ENDBEFR=C'.'),
   %01=(STARTAFT=C'SLDIV',Division
ENDBEFR=C'.',FIXLEN=3),
   %02=(ENDBEFR=C'.',FIXLEN=8),   Order Num
   %03=(ENDBEFR=C'.',FIXLEN=8)),  Doc ID
  OVERLAY=(115:C'Y',
   116:%03,
   124:%01,
   127:%02,
   135:58,08,UFF,M11,LENGTH=7,
   142:135,07,Y4T,TOGREG=Y4T,
   150:X))
*
  OUTFIL VTOF,
  INCLUDE=(115,01,CH,EQ,C'Y'),
  BUILD=(124,03,X,  # Division
 116,08,X,  # Doc ID
 127,08,X,  # Order Num
 19X,   # Filler
 142,08,X,  # Cre Date Greg
 135,07,X)  # Cre Date Jul
/*

For #2, I am trying to create report #1 with a title "Current Order 
Documents/By Division and Document Type". There are no control breaks, 
just a list, showing any items from #1, above, in Ascending Division and 
Order, and Descending Document Createdate.  Here is a sample:

Div OrderDocument  Cre Date
120 $3B37368 DORDP001 20201204
120 @7415755 DPINS003 20201127
120 @7415755 DORDP001 20201129
120 @7415755 DPAYI022 20201204
120 @7415755 DXCHP3R4 20201220
136 $G855813 DORDC244 20210505
242 @Q405827 DPINS003 20200808
242 @Q405827 DPINB022 20200808
242 @Q405827 DORDPP22 20200812

Report #2 is called "Orders With All Documents Current" and is a 
reflection of orders with the most recent (file2) and latest document 
(file1) being the same. There is one record per order.

Div OrderDocument
120 $3B37368 DORDP001
(120 @7415755 is not here bc file2 has DPAYI022, and file1 has a later 
DXCHP3R40 document)

136 $G855813 DORDC244
242 @Q405827 DREFC001

File1 is attached (the LISTCAT output)
File2 a FB 40 file with
120 $3B37368 DORDP001
120 @7415755 DPAYI022
136 $G855813 DORDC244
242 @Q405827 DREFC001
251 DM405862 DORDPP22
251 DM415866 DORDC244
251 DM423871 DINVP002
281 RB415889 DSPL1PR4
281 TA720006 DORDP042
281 TA916078 DPINM111
281 VJ124311 DRET1PR4
288 @@366084 DINVC011
288 @N416086 DRETW023
288 MM737722 DCRMV002

Kolusu, I appreciate whatever guidance you can give me here. I am sure I 
could write a program to do this two-file match, but I am trying to open 
eyes to the possibility of a faster-running SORT process.


Thank you and best regards,
Billy Ashton


-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 12/12/2022 5:02:55 PM
Subject Re: How to SORT Variable records with OVERLAY


 Because my input is a VBA file (out of my control), some of the records are 
much shorter than my SORT FIELDS location.


Billy,

Ideally you can use VLSHRT for padding short records for SORTING. However, if 
you use INREC/OUTREC , it will not work.

So, if you want to use INREC editing you need to PUSH the contents right after 
RDW to retain the variable length records.  If you don't care about variable 
lengths of the output file then use IFOUTLEN= nnn  ( where nnn= max length of 
the overlay/push record).

Alternatively, if you show me the control cards I can show you the optimal way 
of achieving the results.

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.

Re: How to SORT Variable records with OVERLAY

[Sri h Kolusu]

>> Because my input is a VBA file (out of my control), some of the records are 
>> much shorter than my SORT FIELDS location.

Billy,

Ideally you can use VLSHRT for padding short records for SORTING. However, if 
you use INREC/OUTREC , it will not work.  

So, if you want to use INREC editing you need to PUSH the contents right after 
RDW to retain the variable length records.  If you don't care about variable 
lengths of the output file then use IFOUTLEN= nnn  ( where nnn= max length of 
the overlay/push record).

Alternatively, if you show me the control cards I can show you the optimal way 
of achieving the results.

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

How to SORT Variable records with OVERLAY

[Billy Ashton]

Hi all, I am taking a file and using PUSH and OVERLAY to condense 
multiple input lines into a single line. That all works fine and I even 
get the correct format output when I use OPTION COPY or SORT 
FIELDS=COPY.


However, I want to sort on my OVERLAY area fields, but I get an error 
message
ICE218A 3 117 BYTE VARIABLE RECORD IS SHORTER THAN 138 BYTE MINIMUM FOR  
FIELDS


Because my input is a VBA file (out of my control), some of the records 
are much shorter than my SORT FIELDS location.


What do I need to do to use SORT FIELDS=(...) here? Do I need to just 
overlay the unneeded columns at the beginning of the record? All my sort 
keys will be shorter than the shortest record in the file, but I 
expected that by using INCLUDE statements to only keep the long records 
I needed, I would not run into this.


My control cards say
OPTION VLSCMP
SORT FIELDS=(set1,set2, set3)
INCLUDE COND=...,OR, ...
INREC with multiple IFTHEN and PUSH and OVERLAYs
OUTFIL VTOF, INCLUDE,BUILD

There must be something simple I am missing (and there will be another 
question tomorrow if I get this fixed).


Thank you and best regards,
Billy Ashton

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Verwijs, Frederick F [NC]]

Thanks very much Kolusu,

That worked. I'll have to study the parameters so as to understand how to build 
these.


   Regards, 
        Eric Verwijs 

Programmeur-analyste, RPC, SV et solutions de paiement - Direction générale de 
l'innovation, information et technologie
Emploi et Développement social Canada / Gouvernement du Canada
frederick.verw...@hrsdc-rhdcc.gc.ca 
Téléphone 613-415-5465

Programmer Analyst, CPP, OAS, and Payment Solutions - Innovation, Information 
and Technology Branch
Employment and Social Development Canada / Government of Canada
frederick.verw...@hrsdc-rhdcc.gc.ca 
Telephone 613-415-5465

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: November 18, 2022 7:52 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: ICETOOL / SORT capturing count of output records when using INCLUDE

>> Our input file is a tape variable block.

Eric,

Use the following control cards

//SYSINDD *,SYMBOLS=JCLONLY
  OPTION COPY
  INCLUDE COND=(16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'')

  INREC OVERLAY=(1877:16,02,CHANGE=(3,C'',C'100',
C'',C'010',
C'',C'001'),
   NOMATCH=(C'000'))

  OUTFIL FNAMES=OUT1,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=OUT2,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=OUT3,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=REPORT,
  REMOVECC,NODETAIL,VTOF,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
 TOT=(1877,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(1878,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(1879,1,ZD,M10,LENGTH=8))

/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Sri h Kolusu]

>> Our input file is a tape variable block.

Eric,

Use the following control cards

//SYSINDD *,SYMBOLS=JCLONLY
  OPTION COPY
  INCLUDE COND=(16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'')

  INREC OVERLAY=(1877:16,02,CHANGE=(3,C'',C'100',
C'',C'010',
C'',C'001'),
   NOMATCH=(C'000'))

  OUTFIL FNAMES=OUT1,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=OUT2,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=OUT3,INCLUDE=(16,02,CH,EQ,C''),
  VTOF,BUILD=(05,1872)

  OUTFIL FNAMES=REPORT,
  REMOVECC,NODETAIL,VTOF,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
 TOT=(1877,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(1878,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(1879,1,ZD,M10,LENGTH=8))

/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Verwijs, Frederick F [NC]]

Hello Kolusu,

I can't get it to work.

Our input file is a tape variable block.
The data we are selecting on is at byte 16, already adjusted for the 4 bytes.

My output file will be tapes, FB, 1872 bytes. All the data from the input tape.

I get:
ICE027A 9 END OF OUT1 FIELD BEYOND MAXIMUM RECORD LENGTH
ICE751I 0 C5-I90068 C6-I90068 C7-I90068 C8-I90068 E9-I74062 E7-I90068
ICE052I 3 END OF DFSORT

Even when I’m just fiddling with it, using your example, I get the above error 
when I start the field at 5. It works when it starts at 4.

Annoyingly, the above error goes away when I remove the code to get the totals.

What am I missing?

Here's my JCL:
  OPTION COPY
  INCLUDE COND=(16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'',OR,
16,02,CH,EQ,C'')
  INREC BUILD=(05,1872,
   16,2,CHANGE=(3,C'',C'100',
  C'',C'010',
  C'',C'001'),
   NOMATCH=(C'000'))
  OUTFIL FNAMES=SORTOFT1,VTOF,
  OUTREC=(5,1872),
  INCLUDE=(16,2,CH,EQ,C'')
  OUTFIL FNAMES=SORTOFT2,VTOF,
  OUTREC=(5,1872),
  INCLUDE=(16,2,CH,EQ,C'')
  OUTFIL FNAMES=SORTOFT3,VTOF,
  OUTREC=(5,1872),
  INCLUDE=(16,2,CH,EQ,C'')
  OUTFIL FNAMES=REPORT,
 REMOVECC,NODETAIL,BUILD=(80X),
 TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
TOT=(1873,1,ZD,M10,LENGTH=8),/,
   'NUM OF RECORDS FOR TYPE  : ',
TOT=(1874,1,ZD,M10,LENGTH=8),/,
   'NUM OF RECORDS FOR TYPE  : ',
TOT=(1875,1,ZD,M10,LENGTH=8))
/*




   Regards,
Eric Verwijs

Programmeur-analyste, RPC, SV et solutions de paiement - Direction générale de 
l'innovation, information et technologie
Emploi et Développement social Canada / Gouvernement du Canada
frederick.verw...@hrsdc-rhdcc.gc.ca
Téléphone 613-415-5465

Programmer Analyst, CPP, OAS, and Payment Solutions - Innovation, Information 
and Technology Branch
Employment and Social Development Canada / Government of Canada
frederick.verw...@hrsdc-rhdcc.gc.ca
Telephone 613-415-5465


-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: November 17, 2022 5:59 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: ICETOOL / SORT capturing count of output records when using INCLUDE

Eric,

Here is another way to get the output counts into a single dataset.  I also 
used symbols , so that in future if you had to split for a different year, then 
you just need to change the SYMBOLS and the job will automatically take care of 
splitting.

//EXPORT SYMLIST=*
//SET YR01=00
//SET YR02=99
//SET YR03=98
/*
//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC98
DEF98
GHI98
XXX00
YYY00
ZZZ99
//OUT1 DD SYSOUT=*
//OUT2 DD SYSOUT=*
//OUT3 DD SYSOUT=*
//REPORT   DD SYSOUT=*
//SYSINDD *,SYMBOLS=JCLONLY
  OPTION COPY
  INCLUDE COND=(12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'')

  INREC BUILD=(01,20,
   12,2,CHANGE=(3,C'',C'100',
  C'',C'010',
  C'',C'001'),
   NOMATCH=(C'000'))

  OUTFIL FNAMES=OUT1,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT2,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT3,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)

  OUTFIL FNAMES=REPORT,
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
 TOT=(21,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(22,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(23,1,ZD,M10,LENGTH=8))
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with the message: 
INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Verwijs, Frederick F [NC]]

Thanks so much Kolusu. That's perfect and just what we need.

I really appreciate your speedy response.


   Regards, 
        Eric Verwijs 

Programmeur-analyste, RPC, SV et solutions de paiement - Direction générale de 
l'innovation, information et technologie
Emploi et Développement social Canada / Gouvernement du Canada
frederick.verw...@hrsdc-rhdcc.gc.ca 
Téléphone 613-415-5465

Programmer Analyst, CPP, OAS, and Payment Solutions - Innovation, Information 
and Technology Branch
Employment and Social Development Canada / Government of Canada
frederick.verw...@hrsdc-rhdcc.gc.ca 
Telephone 613-415-5465

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: November 17, 2022 5:59 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: ICETOOL / SORT capturing count of output records when using INCLUDE

Eric,

Here is another way to get the output counts into a single dataset.  I also 
used symbols , so that in future if you had to split for a different year, then 
you just need to change the SYMBOLS and the job will automatically take care of 
splitting.

//EXPORT SYMLIST=*
//SET YR01=00
//SET YR02=99
//SET YR03=98
/*
//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC98
DEF98
GHI98
XXX00
YYY00
ZZZ99
//OUT1 DD SYSOUT=*
//OUT2 DD SYSOUT=*
//OUT3 DD SYSOUT=*
//REPORT   DD SYSOUT=*
//SYSINDD *,SYMBOLS=JCLONLY
  OPTION COPY
  INCLUDE COND=(12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'')

  INREC BUILD=(01,20,
   12,2,CHANGE=(3,C'',C'100',
  C'',C'010',
  C'',C'001'),
   NOMATCH=(C'000'))

  OUTFIL FNAMES=OUT1,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT2,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT3,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)

  OUTFIL FNAMES=REPORT,
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
 TOT=(21,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(22,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(23,1,ZD,M10,LENGTH=8))
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Sri h Kolusu]

Eric,

Here is another way to get the output counts into a single dataset.  I also 
used symbols , so that in future if you had to split for a different year, then 
you just need to change the SYMBOLS and the job will automatically take care of 
splitting.

//EXPORT SYMLIST=*
//SET YR01=00
//SET YR02=99
//SET YR03=98
/*
//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC98
DEF98
GHI98
XXX00
YYY00
ZZZ99
//OUT1 DD SYSOUT=*
//OUT2 DD SYSOUT=*
//OUT3 DD SYSOUT=*
//REPORT   DD SYSOUT=*
//SYSINDD *,SYMBOLS=JCLONLY
  OPTION COPY
  INCLUDE COND=(12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'',OR,
12,02,CH,EQ,C'')

  INREC BUILD=(01,20,
   12,2,CHANGE=(3,C'',C'100',
  C'',C'010',
  C'',C'001'),
   NOMATCH=(C'000'))

  OUTFIL FNAMES=OUT1,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT2,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)
  OUTFIL FNAMES=OUT3,INCLUDE=(12,02,CH,EQ,C''),BUILD=(01,20)

  OUTFIL FNAMES=REPORT,
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE  : ',
 TOT=(21,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(22,1,ZD,M10,LENGTH=8),/,
'NUM OF RECORDS FOR TYPE  : ',
 TOT=(23,1,ZD,M10,LENGTH=8))
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: ICETOOL / SORT capturing count of output records when using INCLUDE

[Sri h Kolusu]

>> Can COUNT can be used to determine the total number of records in the output 
>> files? I've not seen how and it seems only able to count the number of input 
>> records.

Eric,

You can split the file into "n" number of files in a single pass of data and it 
is quite simple to generate the output counts also in the same job. Use the 
following Job which will give you the desired results.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
+1+2+3+4+5+---
ABC98
DEF98
GHI98
XXX00
YYY00
ZZZ99
//F00  DD SYSOUT=*
//F00RPT   DD SYSOUT=*
//F98  DD SYSOUT=*
//F98RPT   DD SYSOUT=*
//F99  DD SYSOUT=*
//F99RPT   DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INCLUDE COND=(12,02,CH,EQ,C'00',OR,
12,02,CH,EQ,C'98',OR,
12,02,CH,EQ,C'99')

  INREC BUILD=(01,20)

  OUTFIL FNAMES=F00,INCLUDE=(12,02,CH,EQ,C'00')

  OUTFIL FNAMES=F00RPT,INCLUDE=(12,02,CH,EQ,C'00'),
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE 00 : ',
   COUNT)

  OUTFIL FNAMES=F98,INCLUDE=(12,02,CH,EQ,C'98')

  OUTFIL FNAMES=F98RPT,INCLUDE=(12,02,CH,EQ,C'98'),
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE 98 : ',
   COUNT)

  OUTFIL FNAMES=F99,INCLUDE=(12,02,CH,EQ,C'99')

  OUTFIL FNAMES=F99RPT,INCLUDE=(12,02,CH,EQ,C'99'),
  REMOVECC,NODETAIL,BUILD=(80X),
  TRAILER1=('NUM OF RECORDS FOR TYPE 99 : ',
   COUNT)
/*


Thanks,
Kolusu
DFSORT Development
IBM Corporation

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Eric Verwijs
Sent: Thursday, November 17, 2022 1:51 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] ICETOOL / SORT capturing count of output records when using 
INCLUDE

Hello,

We're splitting very large datasets that have 3 years of client data into 3 
smaller datasets containing a year of data each.

Can COUNT can be used to determine the total number of records in the output 
files? I've not seen how and it seems only able to count the number of input 
records.

I know the job output reports the totals and I could spool that to a temporary 
file to capture it.

However, can COUNT or some other parameter be used to get the total for for 
each of the output files?

This is more or less the JCL that we use:
//SPLITYREXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
-  -  -  -  -  -  -  -  -  -  -  -  -  -
//IN DD *
-  -  -  -  -  -  -  -  -  -  -  -  -  -
/*
-  -  -  -  -  -  -  -  -  -  -  -  -  -
//TOOLIN   DD *
   COPY  FROM(IN) TO(OUT1) USING(CTL1)
   COPY  FROM(IN) TO(OUT2) USING(CTL2)
   COPY  FROM(IN) TO(OUT3) USING(CTL3)
/*
//CTL1CNTL DD *
  OUTFIL FNAMES=OUT1,
 INCLUDE=(12,2,CH,EQ,C'98')
 OUTREC FIELDS=(1:1,20)
/*
//CTL2CNTL DD *
  OUTFIL FNAMES=OUT2,
 INCLUDE=(12,2,CH,EQ,C'99')
 OUTREC FIELDS=(1:1,20)
/*
//CTL3CNTL DD *
  OUTFIL FNAMES=OUT3,
 INCLUDE=(12,2,CH,EQ,C'00')
 OUTREC FIELDS=(1:1,20)
/*

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with the message: 
INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

ICETOOL / SORT capturing count of output records when using INCLUDE

[Eric Verwijs]

Hello,

We're splitting very large datasets that have 3 years of client data into 3 
smaller datasets containing a year of data each.

Can COUNT can be used to determine the total number of records in the output 
files? I've not seen how and it seems only able to count the number of input 
records.

I know the job output reports the totals and I could spool that to a temporary 
file to capture it.

However, can COUNT or some other parameter be used to get the total for for 
each of the output files?

This is more or less the JCL that we use:
//SPLITYREXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*  
//DFSMSG   DD SYSOUT=*  
-  -  -  -  -  -  -  -  -  -  -  -  -  -
//IN DD *   
-  -  -  -  -  -  -  -  -  -  -  -  -  -
/*  
-  -  -  -  -  -  -  -  -  -  -  -  -  -
//TOOLIN   DD * 
   COPY  FROM(IN) TO(OUT1) USING(CTL1)  
   COPY  FROM(IN) TO(OUT2) USING(CTL2)  
   COPY  FROM(IN) TO(OUT3) USING(CTL3)  
/*  
//CTL1CNTL DD * 
  OUTFIL FNAMES=OUT1,   
 INCLUDE=(12,2,CH,EQ,C'98') 
 OUTREC FIELDS=(1:1,20) 
/*  
//CTL2CNTL DD * 
  OUTFIL FNAMES=OUT2,   
 INCLUDE=(12,2,CH,EQ,C'99') 
 OUTREC FIELDS=(1:1,20) 
/* 
//CTL3CNTL DD *
  OUTFIL FNAMES=OUT3,  
 INCLUDE=(12,2,CH,EQ,C'00')
 OUTREC FIELDS=(1:1,20)
/*

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Fixed fields in regex (was: Trying to Parse a LISTCAT with SORT)

[Walt Farrell]

On Tue, 25 Oct 2022 10:33:23 -0500, Paul Gilmartin  wrote:

>On Tue, 25 Oct 2022 08:55:09 -0500, Walt Farrell wrote:

>On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
>>  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
>> ...
>Thanks.  I've wished for something line FIXLEN in regular expressions.

Got an example of what you want in regular expressions?

>>>NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
>>> DATASET-OWNER-(NULL) CREATION2022.200
>>>
>>>becomes:
>>>N04  REPT320220719 MIDDLE  
>>
>>Sorry; doesn't help. I should have been more specific, and started with "what 
>>does FIXLEN do that you'd want to see in a regular expression?"
>>
>
>
>See the ply cited where Sri h Kolusu addressed the OP's requirement.  Could 
>this have been
>done in awk or sed with regular expressions?  In REXX I used LEFT() to achieve 
>the function.

Thanks for the doc reference.

So FIXLEN is applying to the output rather than the input. I don't think there 
are functions like that in standard regular expression processing, since it's 
about input parsing, not creation of output records. I have no idea if sed or 
awk have any specific functions like that for formatting their output, as I 
selecom use them. 
Sorry.

-- 
Walt

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Fixed fields in regex (was: Trying to Parse a LISTCAT with SORT)

[Paul Gilmartin]

On Tue, 25 Oct 2022 08:55:09 -0500, Walt Farrell wrote:
>>>
On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
>  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
> ...
Thanks.  I've wished for something line FIXLEN in regular expressions.
>>>
>>>Got an example of what you want in regular expressions?
>>>
>>NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
>> DATASET-OWNER-(NULL) CREATION2022.200
>>
>>becomes:
>>N04  REPT320220719 MIDDLE  
>
>Sorry; doesn't help. I should have been more specific, and started with "what 
>does FIXLEN do that you'd want to see in a regular expression?"
>


See the ply cited where Sri h Kolusu addressed the OP's requirement.  Could 
this have been
done in awk or sed with regular expressions?  In REXX I used LEFT() to achieve 
the function.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Fixed fields in regex (was: Trying to Parse a LISTCAT with SORT)

[Walt Farrell]

On Sun, 23 Oct 2022 09:36:49 -0500, Paul Gilmartin  wrote:

>On Sun, 23 Oct 2022 08:59:09 -0500, Walt Farrell wrote:
>>
>>>On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
 ...
>>>Thanks.  I've wished for something line FIXLEN in regular expressions.
>>
>>Got an example of what you want in regular expressions?
>>
>NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
> DATASET-OWNER-(NULL) CREATION2022.200
>
>becomes:
>N04  REPT320220719 MIDDLE  

Sorry; doesn't help. I should have been more specific, and started with "what 
does FIXLEN do that you'd want to see in a regular expression?"

-- 
Walt

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Fixed fields in regex (was: Trying to Parse a LISTCAT with SORT)

[Paul Gilmartin]

On Sun, 23 Oct 2022 08:59:09 -0500, Walt Farrell wrote:
>
>>On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
>>>  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
>>> ...
>>Thanks.  I've wished for something line FIXLEN in regular expressions.
>
>Got an example of what you want in regular expressions?
>
NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
 DATASET-OWNER-(NULL) CREATION2022.200

becomes:
N04  REPT320220719 MIDDLE  

>(Possibly best to start a separate thread/topic, though :) )
>
(Continued thread; changed Subject:)

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Trying to Parse a LISTCAT with SORT

[Walt Farrell]

On Sat, 22 Oct 2022 10:03:49 -0500, Paul Gilmartin  wrote:

>On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
>>...
>>  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
>> ...
>Thanks.  I've wished for something line FIXLEN in regular expressions.

Got an example of what you want in regular expressions?

(Possibly best to start a separate thread/topic, though :) )

-- 
Walt

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Trying to Parse a LISTCAT with SORT

[Sri h Kolusu]

>> But:  With  decimal point? without decimal point?

Gil,

DFSORT can process with or without the decimal point. Using the format UFF we 
strip off the decimal point.  UFF (unsigned free form numeric) format extracts 
decimal digits (0-9) from right to left anywhere in the field to form a 
positive number. Any
combination of characters is valid, but characters other than 0-9 are ignored.  
So, I first removed the decimal point using 58,08,UFF,M11,LENGTH=7,  and then 
converted the date to Gregorian format

>> 4-digit year? 2-digit year?

DFSORT can convert both 4 digit and 2 digit years. Here is the table which 
shows the TOJUL and TOGREG output date fields

https://www.ibm.com/docs/en/zos/2.5.0?topic=statements-outfil-control#oufcst__tatodf

>> Sliding window centered on current year?

DFSORT prints a message ICE168I about the sliding window when processing the 2 
digit years.

https://www.ibm.com/docs/en/zos/2.5.0?topic=messages-ice168i

We also provide a runtime option to override it called Y2PAST which can set the 
sliding year

https://www.ibm.com/docs/en/zos/2.5.0?topic=ssc-example-6


Thanks,
Kolusu

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Variable Sort Parsing

[Billy Ashton]

Kolusu, this worked great! I went with the Version 2 selection, and 
replaced the pipe delimiter with just a simple space.


Thanks again - one day, I hope to be able to do these simple things 
myself!


Thank you and best regards,
Billy Ashton


-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 10/20/2022 12:15:22 AM
Subject Re: Variable Sort Parsing


2. The input is a FB file of 150, and these records are just space padded. The 
other log records in the file are of all different lengths, with some using the 
whole 150.



Billy,

It is quite simple to parse out the contents.  I am going to 2 show different 
versions.

Version 1 : will Parse ALL fields and then we pick and choose which fields we 
need
Version 2 : Will Parse ONLY specific fields of interest.

FIXLEN is the parm which determines the length of the parsed field, so adjust 
it according to your needs.

I added a Delimiter of '|' to distinguish the fields. You can remove it if you 
don't need it

Version 1 :

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DISP=SHR,DSN=your Input FB 150 byte file
//SORTOUT  DD SYSOUT=*
//SYSINDD *
 OPTION COPY
 INCLUDE COND=(001,150,SS,EQ,C'MSG004CI')
 INREC IFTHEN=(WHEN=INIT,
BUILD=(001,150,SQZ=(SHIFT=LEFT,MID=C' '))),
   IFTHEN=(WHEN=INIT,
PARSE=(%01=(ENDBEFR=BLANKS,FIXLEN=08),  # Job Time
   %02=(ENDBEFR=BLANKS,FIXLEN=08),  # Job num
   %03=(ENDBEFR=BLANKS,FIXLEN=08),  # Job name
   %04=(ENDBEFR=BLANKS,FIXLEN=08),  # Msg number
   %05=(STARTAFT=C'PARM ',
 ENDBEFR=C': ',FIXLEN=15),  # Prog parm
   %06=(ENDBEFR=C',',FIXLEN=08),# Job Date
   %07=(STARTAFT=C'(',
 ENDBEFR=C') ',FIXLEN=10),  # Run Type
   %08=(ENDBEFR=BLANKS,FIXLEN=15),  # Location
   %09=(ENDBEFR=BLANKS,FIXLEN=06)), # Indicator

BUILD=(%03, # Job name
   C'|',# Delimiter
   %05, # Prog Parm
   C'|',# Delimiter
   %06, # Job Date
   C'|',# Delimiter
   %07, # Run Type
   C'|',# Delimiter
   %08, # Location
   C'|',# Delimiter
   %09, # Indicator
   C'|'))   # Delimiter
/*
The output from this is

PAYW015A|SCHEDULED  |20221008|WEEKLY|ALLSITES   |10.4  |
PAYW052X|REQUEST|20221012|DAILY |OH03   |10.6  |
BILLW015|new|20221016|hourly|ASH01  |10.2  |

Version 2:  Notice the ignored fields with % and pay attention to FIXLEN parm

//SYSINDD *
 OPTION COPY
 INCLUDE COND=(001,150,SS,EQ,C'MSG004CI')
 INREC IFTHEN=(WHEN=INIT,
BUILD=(001,150,SQZ=(SHIFT=LEFT,MID=C' '))),
   IFTHEN=(WHEN=INIT,
PARSE=(%=(ENDBEFR=BLANKS),  # Job Time
   %=(ENDBEFR=BLANKS),  # Job num
   %03=(ENDBEFR=BLANKS,FIXLEN=08),  # Job name
   %=(ENDBEFR=BLANKS),  # Msg number
   %05=(STARTAFT=C'PARM ',
 ENDBEFR=C': ',FIXLEN=03),  # Prog parm
   %06=(ENDBEFR=C',',FIXLEN=08),# Job Date
   %07=(STARTAFT=C'(',
 ENDBEFR=C') ',FIXLEN=01),  # Run Type
   %08=(ENDBEFR=BLANKS,FIXLEN=15),  # Location
   %09=(ENDBEFR=BLANKS,FIXLEN=06)), # Indicator

BUILD=(%03, # Job name
   C'|',# Delimiter
   %05, # Prog Parm
   C'|',# Delimiter
   %06, # Job Date
   C'|',# Delimiter
   %07, # Run Type
   C'|',# Delimiter
   %08, # Location
   C'|',# Delimiter
   %09, # Indicator
   C'|'))   # Delimiter
/*

The output from this is

PAYW015A|SCH|20221008|W|ALLSITES   |10.4  |
PAYW052X|REQ|20221012|D|OH03   |10.6  |
BILLW015|new|20221016|h|ASH01  |10.2  |

Thanks,
Kolusu
DFSORT Development
IBM Corporation




--
For IBM-MAIN subscribe / signoff / archive access in

Re: Trying to Parse a LISTCAT with SORT

[Paul Gilmartin]

On Sat, 22 Oct 2022 04:09:43 +, Sri h Kolusu  wrote:
>...
>  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
> ...
Thanks.  I've wished for something line FIXLEN in regular expressions.


On Sat, 22 Oct 2022 04:17:59 +, Sri h Kolusu wrote:
>...
>It is quite simple to convert the Julian date to Gregorian date using DFSORT.
>All you need is another IFHTEN statement to process the julian date
>
>IFTHEN=(WHEN=(42,10,CH,EQ,C'CREATION--'),
> OVERLAY=(165:58,08,UFF,M11,LENGTH=7,
>  175:165,07,Y4T,TOGREG=Y4T))
>
But:
With  decimal point?
without decimal point?
4-digit year?
2-digit year?
Sliding window centered on current year?

So: 
(It's fairly unbiased -- almost everyone doesn't use it.)

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Trying to Parse a LISTCAT with SORT

[Sri h Kolusu]

>Am I reasonably close? (I don't think so...) Bonus points would be given to 
>convert the date from .DDD to MMDD!

Gil,

It is quite simple to convert the Julian date to Gregorian date using DFSORT.

All you need is another IFHTEN statement to process the julian date

IFTHEN=(WHEN=(42,10,CH,EQ,C'CREATION--'),
 OVERLAY=(165:58,08,UFF,M11,LENGTH=7,
  175:165,07,Y4T,TOGREG=Y4T))


Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Trying to Parse a LISTCAT with SORT

[Sri h Kolusu]

>>. I am trying to use SORT to process a LISTCAT to give me part of the DSN and 
>>the creation date

Don,

Use the following Untested DFSORT control cards which will give you the desired 
results.

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DISP=SHR,DSN=Your Input Listcat VBA 137 byte file
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION COPY,VLSCMP
  INREC IFTHEN=(WHEN=(06,07,CH,EQ,C'NONVSAM'),
   PARSE=(%=(ENDBEFR=C'.',
  ABSPOS=21),# Node 1
  %=(ENDBEFR=C'.'),  # Node 2
  %03=(ENDBEFR=C'.',FIXLEN=8),   # Node 3
  %04=(ENDBEFR=C'.',FIXLEN=8),   # Node 4
  %05=(ENDBEFR=C'.',FIXLEN=8)),  # Node 5
 OVERLAY=(138:%03,
  %04,
  %05))

  OUTREC IFTHEN=(WHEN=GROUP,
BEGIN=(06,07,CH,EQ,C'NONVSAM'),
 PUSH=(138:138,24))

  OUTFIL VTOF,
  INCLUDE=(42,10,CH,EQ,C'CREATION--'),
  BUILD=(146,08,X,
 154,08,X,
 058,08,X,
 138,08,X)
/*

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Trying to Parse a LISTCAT with SORT

[Doug]

Message me off line and I will share what works best for what we need.
No pretty enough to post here but gets what I need for info 
Best Regards 
Doug


.

On Oct 21, 2022, at 19:31, Paul Gilmartin 
<042bfe9c879d-dmarc-requ...@listserv.ua.edu> wrote:

On Fri, 21 Oct 2022 16:22:56 -0500, Don Johnson  wrote:

> Hello! I just tried to post a question with embedded code and fixed 
> formatting, but it was rejected because HTML is not supported.
> 
I tried a HTTP attachment.  LISTSERV ignored it.

> Am I reasonably close? (I don't think so...) Bonus points would be given to 
> convert the date from .DDD to MMDD!
> 
With desktop Regina (ANSI Rexx):
#! /bin/sh

# Test data:
echo "\
NONVSAM --- MYFILE.ABLA.LONGNODE.NODE004.VARNODE1.NOTUSE
 DATASET-OWNER-(NULL) CREATION2018.341
NONVSAM --- MYFILE.ABLA.SHORT.NODE4.RPT2.NOTUSE
 DATASET-OWNER-(NULL) CREATION2015.113
NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
 DATASET-OWNER-(NULL) CREATION2022.200
NONVSAM --- MYFILE.ABLA.SHORT.NODE04.RR.NOTUSE
 DATASET-OWNER-(NULL) CREATION2021.026 " |

rexx /dev/fd/3 3<

Re: Trying to Parse a LISTCAT with SORT

[Paul Gilmartin]

On Fri, 21 Oct 2022 16:22:56 -0500, Don Johnson  wrote:

>Hello! I just tried to post a question with embedded code and fixed 
>formatting, but it was rejected because HTML is not supported.
> 
I tried a HTTP attachment.  LISTSERV ignored it.

>Am I reasonably close? (I don't think so...) Bonus points would be given to 
>convert the date from .DDD to MMDD!
> 
With desktop Regina (ANSI Rexx):
#! /bin/sh

# Test data:
echo "\
NONVSAM --- MYFILE.ABLA.LONGNODE.NODE004.VARNODE1.NOTUSE
  DATASET-OWNER-(NULL) CREATION2018.341
NONVSAM --- MYFILE.ABLA.SHORT.NODE4.RPT2.NOTUSE
  DATASET-OWNER-(NULL) CREATION2015.113
NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
  DATASET-OWNER-(NULL) CREATION2022.200
NONVSAM --- MYFILE.ABLA.SHORT.NODE04.RR.NOTUSE
  DATASET-OWNER-(NULL) CREATION2021.026 " |

rexx /dev/fd/3 3<

Re: Trying to Parse a LISTCAT with SORT

[Paul Gilmartin]

On Fri, 21 Oct 2022 16:22:56 -0500, Don Johnson  
wrote:

>Hello! I just tried to post a question with embedded code and fixed 
>formatting, but it was rejected because HTML is not supported.
>
???
It's supposed to be supported.  Posting from the website, I'll try to attach  
some
HTML.

>So I will post it in plain text, and you can cut/paste into notepad or 3270 to 
>see how it lines up. If there is a way to use code blocks here, please let me 
>know and I won't do this to you again!
> 
???
Just do it!  I view from the LISTSERV website and choose
"Options:   Use Monospaced Font" and it looks fine.  Is your MUA defaulting
to a proportional font?

(more later).

-- 
gil



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Trying to Parse a LISTCAT with SORT

[Don Johnson]

Hello! I just tried to post a question with embedded code and fixed formatting, 
but it was rejected because HTML is not supported.

So I will post it in plain text, and you can cut/paste into notepad or 3270 to 
see how it lines up. If there is a way to use code blocks here, please let me 
know and I won't do this to you again!

Don

Here is the original note...

Hello there. I am trying to use SORT to process a LISTCAT to give me part of 
the DSN and the creation date. I would normally use Rexx for something like 
this, but have over 10 filenames I am working on, and thought this might be 
easier - except I can't get it to work. Part of the problem is that the LISTC 
output is a VBA 137 file, and I have not worked much with VB files. Another 
problem is that the data I want is on two different lines. I can strip the file 
down to these two lines, but can't seem to pull off the individual nodes. Here 
is a sample of how my input file looks for those two lines (without the RDW).

+1+2+3+4+5+6-
NONVSAM --- MYFILE.ABLA.LONGNODE.NODE004.VARNODE1.NOTUSE
   DATASET-OWNER-(NULL) CREATION2018.341
NONVSAM --- MYFILE.ABLA.SHORT.NODE4.RPT2.NOTUSE
   DATASET-OWNER-(NULL) CREATION2015.113
NONVSAM --- MYFILE.ABLA.MIDDLE.N04.REPT3.NOTUSE
   DATASET-OWNER-(NULL) CREATION2022.200
NONVSAM --- MYFILE.ABLA.SHORT.NODE04.RR.NOTUSE
   DATASET-OWNER-(NULL) CREATION2021.026 

The fields in color are the ones I want to have, and as you can see the purple 
(3rd), red (4th), and green (5th) nodes are variable length.

The output should be one line per file, like this:

+1+2+3+4
NODE004  VARNODE1 2018.341 LONGNODE 
NODE4RPT2 2015.113 SHORT
N04  REPT32022.200 MIDDLE   
NODE04   RR   2021.026 SHORT

Here is my mess of code, which gives errors about RDW and just doesn't work.

  OPTION COPY,VLSCMP   
  INCLUDE COND=(001,60,SS,EQ,C'NONVSAM ---',OR,
001,60,SS,EQ,C'CREATION')  
  INREC IFOUTLEN=140,  
 IFTHEN=(WHEN=(06,9,CH,EQ,C'NONVSAM -'),   
  PARSE=(%01=(STARTAFT=C'ABLA.',ENDBEFR=C'.',FIXLEN=8), # NODE3
 %02=(STARTAFT=C'.',ENDBEFR=C'.',FIXLEN=8), # NODE4
 %03=(STARTAFT=C'.',ENDBEFR=C'.',FIXLEN=8))),   # NODE5
 IFTHEN=(WHEN=(42,9,CH,EQ,C'CREATION-'),   
  PARSE=(%04=(STARTAFT=C'',ENDBEFR=BLANKS,FIXLEN=8)), #DATE
  BUILD=(%02,X,
 %03,X,
 %04,X,
 %01,  
 80:X))   # BLANK FILL 
  OUTFIL VTOF,INCLUDE=(35,40,SS,EQ,C'CREATION') 
/* 

Am I reasonably close? (I don't think so...) Bonus points would be given to 
convert the date from .DDD to MMDD!

Thanks for your help!

Don

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Variable Sort Parsing

[Sri h Kolusu]

>>2. The input is a FB file of 150, and these records are just space padded. 
>>The other log records in the file are of all different lengths, with some 
>>using the whole 150.


Billy,

It is quite simple to parse out the contents.  I am going to 2 show different 
versions.

Version 1 : will Parse ALL fields and then we pick and choose which fields we 
need
Version 2 : Will Parse ONLY specific fields of interest.

FIXLEN is the parm which determines the length of the parsed field, so adjust 
it according to your needs.

I added a Delimiter of '|' to distinguish the fields. You can remove it if you 
don't need it

Version 1 :

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DISP=SHR,DSN=your Input FB 150 byte file
//SORTOUT  DD SYSOUT=*
//SYSINDD *
  OPTION COPY
  INCLUDE COND=(001,150,SS,EQ,C'MSG004CI')
  INREC IFTHEN=(WHEN=INIT,
 BUILD=(001,150,SQZ=(SHIFT=LEFT,MID=C' '))),
IFTHEN=(WHEN=INIT,
 PARSE=(%01=(ENDBEFR=BLANKS,FIXLEN=08),  # Job Time
%02=(ENDBEFR=BLANKS,FIXLEN=08),  # Job num
%03=(ENDBEFR=BLANKS,FIXLEN=08),  # Job name
%04=(ENDBEFR=BLANKS,FIXLEN=08),  # Msg number
%05=(STARTAFT=C'PARM ',
  ENDBEFR=C': ',FIXLEN=15),  # Prog parm
%06=(ENDBEFR=C',',FIXLEN=08),# Job Date
%07=(STARTAFT=C'(',
  ENDBEFR=C') ',FIXLEN=10),  # Run Type
%08=(ENDBEFR=BLANKS,FIXLEN=15),  # Location
%09=(ENDBEFR=BLANKS,FIXLEN=06)), # Indicator

 BUILD=(%03, # Job name
C'|',# Delimiter
%05, # Prog Parm
C'|',# Delimiter
%06, # Job Date
C'|',# Delimiter
%07, # Run Type
C'|',# Delimiter
%08, # Location
C'|',# Delimiter
%09, # Indicator
C'|'))   # Delimiter
/*
The output from this is

PAYW015A|SCHEDULED  |20221008|WEEKLY|ALLSITES   |10.4  |
PAYW052X|REQUEST|20221012|DAILY |OH03   |10.6  |
BILLW015|new|20221016|hourly|ASH01  |10.2  |

Version 2:  Notice the ignored fields with % and pay attention to FIXLEN parm

//SYSINDD *
  OPTION COPY
  INCLUDE COND=(001,150,SS,EQ,C'MSG004CI')
  INREC IFTHEN=(WHEN=INIT,
 BUILD=(001,150,SQZ=(SHIFT=LEFT,MID=C' '))),
IFTHEN=(WHEN=INIT,
 PARSE=(%=(ENDBEFR=BLANKS),  # Job Time
%=(ENDBEFR=BLANKS),  # Job num
%03=(ENDBEFR=BLANKS,FIXLEN=08),  # Job name
%=(ENDBEFR=BLANKS),  # Msg number
%05=(STARTAFT=C'PARM ',
  ENDBEFR=C': ',FIXLEN=03),  # Prog parm
%06=(ENDBEFR=C',',FIXLEN=08),# Job Date
%07=(STARTAFT=C'(',
  ENDBEFR=C') ',FIXLEN=01),  # Run Type
%08=(ENDBEFR=BLANKS,FIXLEN=15),  # Location
%09=(ENDBEFR=BLANKS,FIXLEN=06)), # Indicator

 BUILD=(%03, # Job name
C'|',# Delimiter
%05, # Prog Parm
C'|',# Delimiter
%06, # Job Date
C'|',# Delimiter
%07, # Run Type
C'|',# Delimiter
%08, # Location
C'|',# Delimiter
%09, # Indicator
C'|'))   # Delimiter
/*

The output from this is

PAYW015A|SCH|20221008|W|ALLSITES   |10.4  |
PAYW052X|REQ|20221012|D|OH03   |10.6  |
BILLW015|new|20221016|h|ASH01  |10.2  |

Thanks,
Kolusu
DFSORT Development
IBM Corporation




--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Variable Sort Parsing

[Billy Ashton]

Hi Kolusu, and thanks for looking into this.
1. No, these are two different records, with different parameters on 
them. They are both MSG004CI, but have different data fields. I tried to 
show just two with the shortest and the longest length. Each message is 
one line with:
time Jobnum jobname Msgkey - "Program Parm" type date , runtype() 
location numeric-code


2. The input is a FB file of 150, and these records are just space 
padded. The other log records in the file are of all different lengths, 
with some using the whole 150.


Thank you and best regards,
Billy Ashton


-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 10/19/2022 8:57:22 PM
Subject Re: Variable Sort Parsing


This will be fed into another reporting program to generate different reports, 
along with other operational data. Here are the two basic kinds of formats:
 01.34.15 JOB47986  PAYW015A MSG004CI - PROGRAM PARM SCHEDULED: 20221008, 
RUNTYPE(WEEKLY) ALLSITES  10.4

>> 01.34.15 JOB47986  PAYW052X MSG004CI - PROGRAM PARM REQUEST: 20221012,

RUNTYPE(DAILY) OH03  10.6


Hi Billy,

You have the right idea of using PARSE in DFSORT to extract the data. However, 
I need a few more details.

1.  The sample input that you have shown for EACH MSG004CI is split into 2 
records. Is that right ?  If so, can there be a single line message for 
MSG004CI ?
2.  What is the LRECL and RECFM of the input file?

Thanks,
Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Variable Sort Parsing

[Sri h Kolusu]

>> This will be fed into another reporting program to generate different 
>> reports, along with other operational data. Here are the two basic kinds of 
>> formats:
>>  01.34.15 JOB47986  PAYW015A MSG004CI - PROGRAM PARM SCHEDULED: 
>> 20221008, RUNTYPE(WEEKLY) ALLSITES  10.4
 >> 01.34.15 JOB47986  PAYW052X MSG004CI - PROGRAM PARM REQUEST: 20221012,
>>RUNTYPE(DAILY) OH03  10.6

Hi Billy,

You have the right idea of using PARSE in DFSORT to extract the data. However, 
I need a few more details.

1.  The sample input that you have shown for EACH MSG004CI is split into 2 
records. Is that right ?  If so, can there be a single line message for 
MSG004CI ?
2.  What is the LRECL and RECFM of the input file?

Thanks,
Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Variable Sort Parsing

[Billy Ashton]

Hi all, I have a SORT problem I can't quite get my head around. I think 
I want to use PARSE, but can't figure out if there is an easy way to do 
it.


I have an application log file that contains joblogs of thousands of 
jobs a day, all with different records. I am interested in only getting 
records with a key value of MSG004CI . However, the records have 
variable format, and I would like to have SORT not only extract the 
records, but also just give me the fields I need. This will be fed into 
another reporting program to generate different reports, along with 
other operational data. Here are the two basic kinds of formats:
 01.34.15 JOB47986  PAYW015A MSG004CI - PROGRAM PARM SCHEDULED: 
20221008, RUNTYPE(WEEKLY) ALLSITES  10.4
 01.34.15 JOB47986  PAYW052X MSG004CI - PROGRAM PARM REQUEST: 20221012, 
RUNTYPE(DAILY) OH03  10.6


Here are my requirements:
1. We will extract if the message is MSG004CI and then I need the 
jobname word that comes before that.
2. Next, I need to know how the job was run - either Scheduled or 
Request and the date it was run (follows the type). Output is either SCH 
or REQ then the date.
3. Then, I want to know what type of run. This can be Daily, Weekly, 
Monthly, or Yearly. Output D, W, M, or Y.
4. Next, I need to know the locations processed: This is a variable 
length value, ALLSITES is the longest, OH03 is the shortest, with others 
of other lengths between.

5. Finally, I need the numeric indicator at the end of the line.

So given the above records, my output would be
PAYW015A SCH 20221008 W ALLSITES 10.4
PAYW052X REQ 20221012 D OH03 10.6spaces to allow me to have a fixed format>


I think I would use Parse to capture these 6 fields, but don't know how 
to allow for the different positions they could have. Can you help with 
the Inrec definition?


Thank you and best regards,
Billy Ashton

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Optimizing DFSORT Use of Z Sort Accelerator

[kekronbekron]

Hi Sri,

Looking at my TechDocs like, it does have 'DA900' in it; so it must be the same.

What you've shown is that we have to search for something, and then find out 
that there's something new.
What I mean is a very simple page like 
https://public.dhe.ibm.com/s390/newfunctionapars/mvsstore.zosallfu.html, which 
I can visit once in a while to learn the new stuff.
This is what TechDocs used to be...

- KB

--- Original Message ---
On Monday, August 15th, 2022 at 8:44 PM, Sri h Kolusu  
wrote:


> > > I was hoping this would show up in the TechDocs page
>
>
> KB,
>
> Not sure if DA900 translates to "Tech doc". The new page is published as a 
> part of DFSORT, you can find it here
>
> https://www.ibm.com/support/pages/ibmsearch?q=dfsort=desc
>
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Optimizing DFSORT Use of Z Sort Accelerator

[Sri h Kolusu]

>> I was hoping this would show up in the TechDocs page

KB,

Not sure if DA900 translates to "Tech doc".  The new page is published as a 
part of DFSORT, you can find it here 

https://www.ibm.com/support/pages/ibmsearch?q=dfsort=desc


Thanks,
Kolusu
DFSORT Development
IBM Corporation

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Optimizing DFSORT Use of Z Sort Accelerator

[kekronbekron]

Hello Sri,

As always, thanks for your tireless help on this topic.

I was hoping this would show up in the TechDocs page 
(https://www.ibm.com/support/pages/ibmsearch?q==DA900==19380912=desc)
... but of course it doesn't.

Serious question to IBM -
What is the thought process involved in allowing technical folks to produce 
technical content, but then making it impossible to hear about.
Was the gain of re-doing the TechDocs so profound that it dwarfs the impact of 
your customers missing timely technical publications?
In this case, since it's in the newsletter & because of your post, we learn.
What about all the content that goes totally unnoticed?
Ex: How does one really know that this exist - 
https://www.ibm.com/support/pages/system/files/inline-files/zOS%20V2R5%20Communications%20Server%20Performance%20Summary%20Report_3.pdf

- KB

--- Original Message ---
On Monday, August 15th, 2022 at 7:16 PM, Sri h Kolusu  
wrote:


> Hi,
>
> There has been a growing need to get the maximum value from the Z Sort 
> Accelerator on z15s. To address this need, we created three sample ICETOOL 
> reports that analyze DFSORT's SMF type 16 records. we also provided DFSORT 
> Symbol mappings for SMF record type 16.
>
> You can access all of this information from
>
> https://www.ibm.com/support/pages/node/6603051
>
> The information on this page will help you quickly and easily determine how 
> your rollout of Z Sort Accelerator is progressing, and if there are changes 
> that you can make to let even more sorts benefit from Z Sort.
>
> Thanks to Frank Kyne and Cheryl Watson for sharing the reprint of the Cheryl 
> Watson Tuning Letter. The article "Optimizing DFSORT Use of Z Sort 
> Accelerator" answers your questions to optimize your use of Z Sort.
>
> Thanks,
> Kolusu
> DFSORT Development
> IBM Corporation
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Optimizing DFSORT Use of Z Sort Accelerator

[Sri h Kolusu]

Hi,

There has been a growing need to get the maximum value from the Z Sort 
Accelerator on z15s. To address this need, we created three sample ICETOOL 
reports that analyze DFSORT's SMF type 16 records. we also provided DFSORT 
Symbol mappings for SMF record type 16.

You can access all of this information from

https://www.ibm.com/support/pages/node/6603051

The information on this page will help you quickly and easily determine how 
your rollout of Z Sort Accelerator is progressing, and if there are changes 
that you can make to let even more sorts benefit from Z Sort.

Thanks to Frank Kyne and Cheryl Watson for sharing the reprint of the Cheryl 
Watson Tuning Letter. The article "Optimizing DFSORT Use of Z Sort Accelerator" 
 answers your questions to optimize your use of Z Sort.

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Using SORT to generate sequential Dates

[Billy Ashton]

Hi Kolusu, and thanks for your help. I have been out for a few days, and 
got to work with it today. It is just what I need, and is easy enough to 
change the volume of data to add another month (which I was going to try 
later).


I appreciate all that you do, and for your expert knowledge!

Thank you and best regards,
Billy Ashton

-- Original Message --

From "Sri h Kolusu" 

To IBM-MAIN@listserv.ua.edu
Date 6/30/2022 6:14:04 PM
Subject Re: Using SORT to generate sequential Dates


Billy,

If I understand your requirement correctly, you need current month begin to end 
and 1 or 2 months before the current month. So if you are run the job 
today(June 30th), it will generate the dates from April 1st to June 30th.   If 
you run the job on July 1st then it would generate dates beginning May 1 thru 
May 31 , June 1 thru June 30 and July 1 thru July 31. If that is true, then use 
the following job which will give you the desired results.

//***
//* Generate DFSORT symbols for begin date and repeat value *
//* Begindate  =  Current month - 2 months (CCYYMM)  01 *
//* Enddate=  Last day of the current month (CCYYMMDD)  *
//* Repeatval  =  Number of entries (Enddate - Begmmdd) *
//***
//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD DSN=&,DISP=(,PASS),SPACE=(TRK,(1,0))
//SYSINDD *
 OPTION COPY,STOPAFT=1,NULLOUT=RC4
 INREC OVERLAY=(01:DATE2-2,C'01',
11:DATE1,
21:11,08,Y4T,LASTDAYM,TOGREG=Y4T,
31:21,08,Y4T,DATEDIFF,01,08,Y4T)

 OUTFIL BUILD=(C'SDATE,C''',01,08,C,/,
   C'EDATE,C''',21,08,C,/,
   C'RPVAL,',31,08,UFF,ADD,+1,M11,LENGTH=8,
   80:X)
/*
//***
//* Use the symbols from previous step and generate the data*
//***
//STEP0200 EXEC PGM=SORT,COND=(0,LT)
//SYSOUT   DD SYSOUT=*
//SYMNOUT  DD SYSOUT=*
//SYMNAMES DD DISP=(OLD,PASS),DSN=&
//SORTIN   DD *
ABC
//SORTOUT  DD SYSOUT=*
//SYSINDD *
 OPTION COPY,STOPAFT=1,NULLOUT=RC4
 OUTFIL REPEAT=RPVAL,
 IFOUTLEN=80,
 IFTHEN=(WHEN=INIT,
  OVERLAY=(081:SDATE,
   091:SEQNUM,3,ZD,START=0,
   101:081,08,Y4T,ADDDAYS,091,3,ZD,TOGREG=Y4T)),
 IFTHEN=(WHEN=INIT,
 BUILD=(C'(''',C'D',SEQNUM,3,ZD,C''',',
C,101,04,C'-',105,02,C'-',107,02,C''',',
C,101,08,C''',',
101,08,C',',
C,101,02,C''',',
C,103,02,C''',',
C,105,02,C''',',
C,107,02,C''')'))
/*


Thanks,
Sri Hari Kolusu
DFSORT Development
IBM Corporation


-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Billy Ashton
Sent: Thursday, June 30, 2022 2:30 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Using SORT to generate sequential Dates

Kolusu, the key data is that I have the previous month start and end date (for 
today, it would be May 1 and 31), with some range of dates before that. I also 
need at least the end of the current month (June 30, today). For running it 
tomorrow, I need at least June 1 - 30 (with some dates in May) and July 1 to 31.
Having more dates than this is fine, and is no problem to go back even another 
month and ahead even another month.

Billy

Sri h Kolusu wrote:






I need this for about 3 months worth of dates. . The dates should
start either on the 15th of the month, 2 months ago, or 45 days ago
(whichever is easier)





Billy,

It is easy to generate the dates in the required format, however  a
few clarifications needed.
•   Can the dates start from current date and go back to 3 months
of dates?
•   Do you want the ability of passing ANY VALID date and number
of entries needed?
•   3 months dates can result in 88 - 93 entries depending on the
current date. For example today's  Current-date is 2022-06-30 and
subtracting 3 months from it results in 2022-03-30. So we can have the
start date from March 30th and get to Current date which will be 93
days of data. Is that OK? If we always use 93 as number of entries to
generate then it might exceed the current date.  I can handle it
dynamically have the repeat value. Is that needed ? or is it ok to
have 93 entries of data?


Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions, send
email to lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with the 
message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> wi