Re: Can I do variable field-length blanking with SORT?
Sri, stunning as usual ! ICETOOL is really terrific . and I've learnt much more (CHANGE for instance). Thanks a lot. Max Il giorno gio 9 gen 2020 alle ore 02:16 Sri h Kolusu ha scritto: > > Please "light us" with the trick. > > Max, > > Here is the trick job I was talking about. Basically you resize the record > into 4 byte chunks and use 4 IFTHEN statements to space out the record. > This job will work for any LRECL 1-(assuming you adjust the pos and > lengths) and if you want to handle LRECL (1 - 32760), then it would be > a 5 byte resize. > > //*** > //** CREATE SAMPLE DATA WITH LRECL=101 AND RECFM=FB** > //*** > //STEP0050 EXEC PGM=SORT > //SYSOUT DD SYSOUT=* > //SORTIN DD * > H 01 COMMENT: "01" IS SIGNIFICANT DATA COLUMNS > D 12345XYZ ABC DEF > D 45678GHI JKL MNO > H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS > D 12345XYZ ABC DEF > D 45678GHI JKL MNO > H 10 > D 1234567890 ABCDEFGHIJKL > D 9876543210 MNOPQWRSTUVWXYZ > H 98 > Y A234567890 ABCDEFGHIJKL > Y B876543210 MNOPQWRSTUVWXYZ > Y C876543210 MNOPQWRSTUVWXYZ > H 99 > Z A234567890 ABCDEFGHIJKL > Z B876543210 MNOPQWRSTUVWXYZ > Z C876543210 MNOPQWRSTUVWXYZ > //SORTOUT DD DSN=&,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE) > //SYSINDD * > OPTION COPY > INREC IFOUTLEN=101, > IFTHEN=(WHEN=(1,3,CH,EQ,C'Y A'),OVERLAY=(3:98C'A',C'@')), > IFTHEN=(WHEN=(1,3,CH,EQ,C'Y B'),OVERLAY=(3:98C'B',C'#')), > IFTHEN=(WHEN=(1,3,CH,EQ,C'Y C'),OVERLAY=(3:98C'C',C'$')), > IFTHEN=(WHEN=(1,3,CH,EQ,C'Z A'),OVERLAY=(3:99C'X',C'@')), > IFTHEN=(WHEN=(1,3,CH,EQ,C'Z B'),OVERLAY=(3:99C'Y',C'#')), > IFTHEN=(WHEN=(1,3,CH,EQ,C'Z C'),OVERLAY=(3:99C'Z',C'$')) > /* > //*** > //** BLANK OUT THE DETAIL RECORD WITH LENGTH VALUE ON HEADER ** > //*** > //STEP0100 EXEC PGM=ICETOOL > //TOOLMSG DD SYSOUT=* > //DFSMSG DD SYSOUT=* > //INP DD DISP=SHR,DSN=& > //TMP DD DSN=&,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE) > //OUT DD SYSOUT=* > //TOOLIN DD * > RESIZE FROM(INP) TO(TMP) TOLEN(004) USING(CTL1) > RESIZE FROM(TMP) TO(OUT) TOLEN(108) USING(CTL2) > //CTL1CNTL DD * > OMIT COND=(3,2,ZD,LT,1) > INREC IFOUTLEN=108, > IFTHEN=(WHEN=INIT, >BUILD=(C'00',2X,1,101)), > IFTHEN=(WHEN=GROUP,BEGIN=(5,1,CH,EQ,C'H'), > PUSH=(03:7,2)), > IFTHEN=(WHEN=INIT, > OVERLAY=(01:01,4,ZD,ADD,+2,EDIT=(), > 01:5,1,CHANGE=(4,C'H',C'0101'), > NOMATCH=(1,4))) > > OUTFIL IFOUTLEN=4, > IFTHEN=(WHEN=GROUP,RECORDS=27,PUSH=(6:1,4,ID=8)), > IFTHEN=(WHEN=INIT, > OVERLAY=(20:SEQNUM,4,ZD,START=0,INCR=4,RESTART=(6,12), >26:06,4,ZD,SUB,20,4,ZD,ZD,LENGTH=4)), > IFTHEN=(WHEN=(26,4,ZD,EQ,-1),OVERLAY=(4:1X)), > IFTHEN=(WHEN=(26,4,ZD,EQ,-2),OVERLAY=(3:2X)), > IFTHEN=(WHEN=(26,4,ZD,EQ,-3),OVERLAY=(2:3X)), > IFTHEN=(WHEN=(26,4,ZD,LE,-4),OVERLAY=(1:4X)) > /* > //CTL2CNTL DD * > OUTFIL BUILD=(5,101) > /* > > Thanks, > Kolusu > > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Can I do variable field-length blanking with SORT?
> Please "light us" with the trick. Max, Here is the trick job I was talking about. Basically you resize the record into 4 byte chunks and use 4 IFTHEN statements to space out the record. This job will work for any LRECL 1-(assuming you adjust the pos and lengths) and if you want to handle LRECL (1 - 32760), then it would be a 5 byte resize. //*** //** CREATE SAMPLE DATA WITH LRECL=101 AND RECFM=FB** //*** //STEP0050 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD * H 01 COMMENT: "01" IS SIGNIFICANT DATA COLUMNS D 12345XYZ ABC DEF D 45678GHI JKL MNO H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS D 12345XYZ ABC DEF D 45678GHI JKL MNO H 10 D 1234567890 ABCDEFGHIJKL D 9876543210 MNOPQWRSTUVWXYZ H 98 Y A234567890 ABCDEFGHIJKL Y B876543210 MNOPQWRSTUVWXYZ Y C876543210 MNOPQWRSTUVWXYZ H 99 Z A234567890 ABCDEFGHIJKL Z B876543210 MNOPQWRSTUVWXYZ Z C876543210 MNOPQWRSTUVWXYZ //SORTOUT DD DSN=&,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE) //SYSINDD * OPTION COPY INREC IFOUTLEN=101, IFTHEN=(WHEN=(1,3,CH,EQ,C'Y A'),OVERLAY=(3:98C'A',C'@')), IFTHEN=(WHEN=(1,3,CH,EQ,C'Y B'),OVERLAY=(3:98C'B',C'#')), IFTHEN=(WHEN=(1,3,CH,EQ,C'Y C'),OVERLAY=(3:98C'C',C'$')), IFTHEN=(WHEN=(1,3,CH,EQ,C'Z A'),OVERLAY=(3:99C'X',C'@')), IFTHEN=(WHEN=(1,3,CH,EQ,C'Z B'),OVERLAY=(3:99C'Y',C'#')), IFTHEN=(WHEN=(1,3,CH,EQ,C'Z C'),OVERLAY=(3:99C'Z',C'$')) /* //*** //** BLANK OUT THE DETAIL RECORD WITH LENGTH VALUE ON HEADER ** //*** //STEP0100 EXEC PGM=ICETOOL //TOOLMSG DD SYSOUT=* //DFSMSG DD SYSOUT=* //INP DD DISP=SHR,DSN=& //TMP DD DSN=&,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE) //OUT DD SYSOUT=* //TOOLIN DD * RESIZE FROM(INP) TO(TMP) TOLEN(004) USING(CTL1) RESIZE FROM(TMP) TO(OUT) TOLEN(108) USING(CTL2) //CTL1CNTL DD * OMIT COND=(3,2,ZD,LT,1) INREC IFOUTLEN=108, IFTHEN=(WHEN=INIT, BUILD=(C'00',2X,1,101)), IFTHEN=(WHEN=GROUP,BEGIN=(5,1,CH,EQ,C'H'), PUSH=(03:7,2)), IFTHEN=(WHEN=INIT, OVERLAY=(01:01,4,ZD,ADD,+2,EDIT=(), 01:5,1,CHANGE=(4,C'H',C'0101'), NOMATCH=(1,4))) OUTFIL IFOUTLEN=4, IFTHEN=(WHEN=GROUP,RECORDS=27,PUSH=(6:1,4,ID=8)), IFTHEN=(WHEN=INIT, OVERLAY=(20:SEQNUM,4,ZD,START=0,INCR=4,RESTART=(6,12), 26:06,4,ZD,SUB,20,4,ZD,ZD,LENGTH=4)), IFTHEN=(WHEN=(26,4,ZD,EQ,-1),OVERLAY=(4:1X)), IFTHEN=(WHEN=(26,4,ZD,EQ,-2),OVERLAY=(3:2X)), IFTHEN=(WHEN=(26,4,ZD,EQ,-3),OVERLAY=(2:3X)), IFTHEN=(WHEN=(26,4,ZD,LE,-4),OVERLAY=(1:4X)) /* //CTL2CNTL DD * OUTFIL BUILD=(5,101) /* Thanks, Kolusu -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Can I do variable field-length blanking with SORT?
Sri, your view is always important to us. The fix for 99 is not so difficult. Please "light us" with the trick. :D Max Il giorno mer 8 gen 2020 alle ore 15:09 Sri h Kolusu ha scritto: > > this could be a more useful version: > > Massimo, > > Peter mentioned that the input has a 2-byte zoned decimal "data length" > value which identifies the number of SIGNIFICANT columns. So the maximum > value that a 2 byte zoned decimal can hold is 99. Your job fails as the > arithmetic ends with a negative number > > Try this sample data > > H 99 > D > > > D > > > D > > > > > It is an easy fix, however note that your solution might not scalable for > an bigger LRECL's as you are simply generating the IFTHEN statements for > every unique length. Imagine if you have unique lengths from 1 - 32760 > ( OP has only 2 bytes of zoned decimal but what if it was unsigned binary > field? > > There is a trick to get this done without generating IFTHEN statements. > > > Kolusu > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Can I do variable field-length blanking with SORT?
> this could be a more useful version: Massimo, Peter mentioned that the input has a 2-byte zoned decimal "data length" value which identifies the number of SIGNIFICANT columns. So the maximum value that a 2 byte zoned decimal can hold is 99. Your job fails as the arithmetic ends with a negative number Try this sample data H 99 D D D It is an easy fix, however note that your solution might not scalable for an bigger LRECL's as you are simply generating the IFTHEN statements for every unique length. Imagine if you have unique lengths from 1 - 32760 ( OP has only 2 bytes of zoned decimal but what if it was unsigned binary field? There is a trick to get this done without generating IFTHEN statements. Kolusu -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Can I do variable field-length blanking with SORT?
Peter,
this could be a more useful version:
//ST020 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD *
H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS
D 12345 XYZ ABC DEF
D 45678 GHI JKL MNO
H 10
D 1234567890 ABCDEFGHIJKL
D 9876543210 MNOPQWRSTUVWXYZ
/*
//SORTOUT DD DSN=,DISP=(,PASS),SPACE=(TRK,(1,1,),RLSE)
//SYMNAMES DD *
LLREC,80 LLREC
LLREC1,+80 LLREC WITH + SIGN
S1,X'7D'
/*
//SYSIN DD *
INCLUDE COND=(1,1,CH,EQ,C'H')
INREC FIELDS=(3,2,
3,2,ZD,ADD,+3,TO=ZD,LENGTH=2,
LLREC1,SUB,3,2,ZD,TO=ZD,LENGTH=2,
LLREC1,SUB,+0,TO=ZD,LENGTH=2)
SORT FIELDS=(1,2,BI,A)
SUM FIELDS=NONE
OUTFIL FNAMES=SORTOUT,REMOVECC,
OUTREC=(2X,C'IFTHEN=(WHEN=(81,2,CH,EQ,C',S1,
1,2,S1,C'),BUILD=(1,',3,2,C',',5,2,C'X)),',80:X),
TRAILER1=(' IFTHEN=(WHEN=NONE,BUILD=(1,',7,2,'))')
/*
//ST030 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD *
H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS
D 12345 XYZ ABC DEF
D 45678 GHI JKL MNO
H 10
D 1234567890 ABCDEFGHIJKL
D 9876543210 MNOPQWRSTUVWXYZ
/*
//SORTOUT DD SYSOUT=*
//SYSIN DD *
INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,1,CH,EQ,C'H'),
PUSH=(81:3,2))
SORT FIELDS=COPY
OUTREC IFTHEN=(WHEN=(1,1,CH,EQ,C'H'),
BUILD=(1,80)),
// DD DSN=,DISP=(OLD,DELETE)
ST020 analyze only "H" records and builds one "build" statement per every
found length.
At the end (trailer1) inserts the WHEN=NONE statement needed to obtain the
right lrecl in output.
I assumed the value in the H records is always > 3.
Pls, let me know if it worked.
Best regards.
Max
Il giorno mer 8 gen 2020 alle ore 09:50 Massimo Biancucci
ha scritto:
> Peter,
>
> I don't know how long is your record, assuming it's 80 and that the LL
> field in H record is between 1 and 99, this could be a solution:
>
> //SYSIN DD *
> INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,1,CH,EQ,C'H'),
> PUSH=(81:3,2))
> SORT FIELDS=COPY
> OUTREC IFTHEN=(WHEN=(1,1,CH,EQ,C'H'),
> BUILD=(1,80)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'01'),
> OVERLAY=(04:76X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'02'),
> OVERLAY=(05:75X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'03'),
> OVERLAY=(06:74X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'04'),
> OVERLAY=(07:73X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'05'),
> OVERLAY=(08:72X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'06'),
> OVERLAY=(09:71X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'07'),
> OVERLAY=(10:70X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'08'),
> OVERLAY=(11:69X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'09'),
> OVERLAY=(12:68X)),
> IFTHEN=(WHEN=(81,2,CH,EQ,C'10'),
> OVERLAY=(13:67X))
> /*
>
> I stopped writing the IFTHEN at length 10, you can code the other
> statements.
>
> It's possible to "pre-analyze" the "H" records and dynamically generate
> only the needed IFTHEN statements.
>
> This is the output:
>
> H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS
> D 12345
> D 45678
> H 10
> D 1234567890
> D 9876543210
>
> Hope this helps.
>
> Best regards.
> Max
>
> Il giorno mer 8 gen 2020 alle ore 01:34 Farley, Peter x23353 <
> peter.far...@broadridge.com> ha scritto:
>
>> I have a fixed-record-length data file which has "subsections" starting
>> with a "data characteristics" record (call it a "subsection header"). One
>> of the fields in the "subsection header" is a 2-byte zoned decimal "data
>> length" value which identifies the number of SIGNIFICANT columns in the
>> "data" portion of the records following the "subsection header".
>>
>> What I would like to do is to blank out the NON-significant data columns
>> in the data records that follow each "subsection header". The data records
>> have suffered from some "data pollution" where non-significant data has
>> been accidentally stored beyond the significant data columns.
>>
>> Each "subsection header" may have a different "significant data" length
>> value for the following data records.
>>
>> Example INPUT data (column 1 = record type [H = header, D = data], data
>> starts in column 3 in each record):
>>
>> H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS
>> D 12345 XYZ ABC DEF
>> D 45678 GHI JKL MNO
>> H 10
>> D 1234567890 ABCDEFGHIJKL
>> D 9876543210 MNOPQWRSTUVWXYZ
>>
>> Example OUTPUT data (column 1 = record type [H = header, D = data], data
>> starts in column 3):
>>
>> H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS
>> D 12345
>> D 45678
>> H 10
>> D 1234567890
>> D 9876543210
>>
>> Obviously I can write a pretty simple script or program to accomplish
>> this "data cleaning" operation, but I wondered if it would be possible
>> using just SORT.
>>
>> The data volume is in the range of about 100K-200K records per file if
>> that matters.
>>
>> TIA for any ideas you can offer.
>>
>> Peter
>> --
>>
>>
>>
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu
Re: Can I do variable field-length blanking with SORT?
Peter, I don't know how long is your record, assuming it's 80 and that the LL field in H record is between 1 and 99, this could be a solution: //SYSIN DD * INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,1,CH,EQ,C'H'), PUSH=(81:3,2)) SORT FIELDS=COPY OUTREC IFTHEN=(WHEN=(1,1,CH,EQ,C'H'), BUILD=(1,80)), IFTHEN=(WHEN=(81,2,CH,EQ,C'01'), OVERLAY=(04:76X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'02'), OVERLAY=(05:75X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'03'), OVERLAY=(06:74X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'04'), OVERLAY=(07:73X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'05'), OVERLAY=(08:72X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'06'), OVERLAY=(09:71X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'07'), OVERLAY=(10:70X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'08'), OVERLAY=(11:69X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'09'), OVERLAY=(12:68X)), IFTHEN=(WHEN=(81,2,CH,EQ,C'10'), OVERLAY=(13:67X)) /* I stopped writing the IFTHEN at length 10, you can code the other statements. It's possible to "pre-analyze" the "H" records and dynamically generate only the needed IFTHEN statements. This is the output: H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS D 12345 D 45678 H 10 D 1234567890 D 9876543210 Hope this helps. Best regards. Max Il giorno mer 8 gen 2020 alle ore 01:34 Farley, Peter x23353 < peter.far...@broadridge.com> ha scritto: > I have a fixed-record-length data file which has "subsections" starting > with a "data characteristics" record (call it a "subsection header"). One > of the fields in the "subsection header" is a 2-byte zoned decimal "data > length" value which identifies the number of SIGNIFICANT columns in the > "data" portion of the records following the "subsection header". > > What I would like to do is to blank out the NON-significant data columns > in the data records that follow each "subsection header". The data records > have suffered from some "data pollution" where non-significant data has > been accidentally stored beyond the significant data columns. > > Each "subsection header" may have a different "significant data" length > value for the following data records. > > Example INPUT data (column 1 = record type [H = header, D = data], data > starts in column 3 in each record): > > H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS > D 12345 XYZ ABC DEF > D 45678 GHI JKL MNO > H 10 > D 1234567890 ABCDEFGHIJKL > D 9876543210 MNOPQWRSTUVWXYZ > > Example OUTPUT data (column 1 = record type [H = header, D = data], data > starts in column 3): > > H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS > D 12345 > D 45678 > H 10 > D 1234567890 > D 9876543210 > > Obviously I can write a pretty simple script or program to accomplish this > "data cleaning" operation, but I wondered if it would be possible using > just SORT. > > The data volume is in the range of about 100K-200K records per file if > that matters. > > TIA for any ideas you can offer. > > Peter > -- > > > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Can I do variable field-length blanking with SORT?
I have a fixed-record-length data file which has "subsections" starting with a "data characteristics" record (call it a "subsection header"). One of the fields in the "subsection header" is a 2-byte zoned decimal "data length" value which identifies the number of SIGNIFICANT columns in the "data" portion of the records following the "subsection header". What I would like to do is to blank out the NON-significant data columns in the data records that follow each "subsection header". The data records have suffered from some "data pollution" where non-significant data has been accidentally stored beyond the significant data columns. Each "subsection header" may have a different "significant data" length value for the following data records. Example INPUT data (column 1 = record type [H = header, D = data], data starts in column 3 in each record): H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS D 12345 XYZ ABC DEF D 45678 GHI JKL MNO H 10 D 1234567890 ABCDEFGHIJKL D 9876543210 MNOPQWRSTUVWXYZ Example OUTPUT data (column 1 = record type [H = header, D = data], data starts in column 3): H 05 COMMENT: "05" IS SIGNIFICANT DATA COLUMNS D 12345 D 45678 H 10 D 1234567890 D 9876543210 Obviously I can write a pretty simple script or program to accomplish this "data cleaning" operation, but I wondered if it would be possible using just SORT. The data volume is in the range of about 100K-200K records per file if that matters. TIA for any ideas you can offer. Peter -- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
