On Wed, 26 Apr 2017 08:55:29 -0400, Bill Ashton <[email protected]> wrote:
>Hello again! I would like to know what you all are using to access web
>pages in batch JCL.
>
>I have some internal webpages (built from multiple systems) that contain
>particular information that I want to capture in a batch job, and then I
>will combine that with other data from other mainframe files. The easiest
>way is to grab the webpage in m batch job, and then I can use Rexx or Sort
>to parse through the HTML to get what I need.
>
>That's about all I can say on this project due to its sensitive nature.
>Please tell me your thoughts about how you would grab these web pages.
>
>Thanks!
>B
>
>--
>Thank you and best regards,
>*Billy Ashton*
Here is some sample REXX Sockets code that I wrote that you can use as a
starting point. The sample code is based on some real production code that I
wrote several years ago. The code was used to read a file from a web site and
write it to a z/OS data set. The code can also handle redirection headers and
since I use VM/CMS a lot, the code can run in VM/CMS and in TSO, (TSO Batch
from JCL).
/*-------------------------- SAMPHTTP REXX ---------------------------*/
/* Please see "EPILOGUE:" for a complete description of this command. */
/*--------------------------------------------------------------------*/
SAMPHTTP:
Parse Source . ctype ename eddname edsname . system .
Parse Value 0 With =1 cms =1 tso .
version = 1 /* Program Version */
/*--------------------------------------------------------------------*/
/* The following three fields define the Server, File Location, and */
/* File Name for the file to be retrieved via HTTP thru REXX Sockets. */
/* The fourth field is the Path to the file, (the File Location and */
/* File Name combined). */
/*--------------------------------------------------------------------*/
site = 'www.yourserver.com:port' /* Port will Default to 80 */
dir = '/dir1/dir2/...'
file = 'file1.file2'
path = dir||file
/*--------------------------------------------------------------------*/
reclen = nnn /* Output File Record Length */
mvsfile = 'OUTPUT' /* MVS DD Name to Write Data to */
cmsfile = 'OUTPUT DATA' /* CMS File to Write Data to */
cmsfm = 'A' /* Default CMS Filemode */
crlfa = '0D0A'x /* ASCII Carriage Return/Line Feed */
crlfe = '0D25'x /* EBCDIC Carriage Return/Line Feed */
msghdr = ename': '
Select /* system */
When system = 'CMS' Then cms = 1
When system = 'TSO' Then tso = 1
Otherwise
Say msghdr 'Invalid Operating Environment: ' system
Say msghdr ename 'must be invoked from CMS or TSO.'
Exit 12
End /* Select system */
If tso Then
Do
Address TSO
'EXECIO 0 DISKW' mvsfile '(OPEN'
If rc \= 0 Then
Do
Say msghdr 'Error' rc 'Opening MVS File' mvsfile
Say msghdr 'The DD Statement for' mvsfile,
'is Probably Missing'
Exit rc
End
End
Else
Do
Address COMMAND
Arg fm .
If fm = '' Then fm = cmsfm
'CMDCALL VALIDATE * *' fm
If rc \= 0 Then Exit rc
fm1 = Left(fm,1)
cmsfile = cmsfile fm
'ERASE' cmsfile
If rc \= 0 & rc \= 28 Then
Do
Say msghdr 'Error' rc 'No R/W' fm1'-Disk Available!'
Exit rc
End
End
Call SETUP /* Initialize Conversion Strings */
string = GET_FILE(site,path)
Parse Var string . src .
If WordPos(src,'301 302') > 0 Then /* Redirect */
Do
Call REDIRECT
If result > 0 Then Exit result
string = GET_FILE(newsite,newpath)
End
Call STR2STEM string /* Convert String to Stem */
If tso Then
Do
Say msghdr 'Creating MVS File "'mvsfile'"...'
'EXECIO' rec.0 'DISKW' mvsfile '(FINIS STEM REC.'
src = rc
If src \= 0 Then
Say msghdr 'Error' src 'Creating MVS File "'mvsfile'".'
End
Else
Do
Say msghdr 'Creating CMS File "'cmsfile'"...'
'PIPE STEM REC.|>' cmsfile 'F'
src = rc
If src \= 0 Then
Say msghdr 'Error' src 'Creating CMS File "'cmsfile'".'
End
Exit src
GET_FILE: /* Get Requested File from the Server */
Parse Arg server, request, .
Parse Var server server ':' port .
If port = '' Then port = 80
resphdr = 'HTTP/1.1 200 Document follows.'crlfa||crlfa
id = 'SAMPID01'
Say msghdr 'Creating Socket Set...'
Parse Value Socket('INITIALIZE',id) With rc sockid maxdesc service .
If rc \= 0 Then
Do
respmsg = 'Initialization Error' rc':' sockid',',
maxdesc',' service
Say msghdr respmsg
Return resphdr||respmsg
End
Say msghdr 'Opening Socket...'
Parse Value Socket('SOCKET','AF_INET','STREAM',0) With rc socket .
If rc \= 0 Then
Do
respmsg = 'Socket Error' rc':' socket
Say msghdr respmsg
Return resphdr||respmsg
End
Say msghdr 'Connecting to' server'...'
rc = Socket('CONNECT',socket,'AF_INET' port server)
If rc \= 0 Then
Do
respmsg = 'Connect Error' rc':' server
Say msghdr respmsg
Call END_SOCKET
Return resphdr||respmsg
End
message = E2A('GET' request 'HTTP/1.0'crlfe||,
'Host:' server':'port||crlfe||,
'Accept: */*'crlfe||,
'Accept-Language: en-us'crlfe||,
'User-Agent: Mozilla/4.0 (compatible; MSIE 5.0;',
'Windows NT; DigExt)'crlfe||crlfe)
Say msghdr 'Sending Request to' server'...'
Parse Value Socket('SEND',socket,message) With rc length .
If rc \= 0 Then
Do
respmsg = 'Send Error' rc':' server
Say msghdr respmsg
Call END_SOCKET
Return resphdr||respmsg
End
string = ''
Say msghdr 'Receiving Data from' server'...'
Parse Value Socket('RECV',socket) With rc length response
Do While rc = 0 & length > 0
text = A2E(response)
string = string||text
Parse Value Socket('RECV',socket) With rc length response
End /* Do Until rc \= 0 | length = 0 */
Call END_SOCKET
Return string
END_SOCKET: /* Close/Terminate Socket */
Say msghdr 'Closing Socket...'
rc = Socket('CLOSE',socket)
Say msghdr 'Deleting Socket Set...'
rc = Socket('TERMINATE',sockid)
Return
A2E: /* Convert ASCII to EBCDIC */
Return Translate(Arg(1),ascii,hexall)
E2A: /* Convert EBCDIC to ASCII */
Return Translate(Arg(1),ebcdic,hexall)
STR2STEM: /* Convert String to Stem */
Parse Arg string
rec.0 = 0
eoh = Pos(crlfe||crlfe,string)
If eoh = 0 Then Return
string = SubStr(string,eoh+4)
n = 0
eor = Pos(crlfe,string)
Do While eor > 0
rec = SubStr(string,1,eor-1)
If Strip(rec) \= '' Then
Do
n = n + 1
rec.n = Left(rec,reclen)
End
string = SubStr(string,eor+2)
eor = Pos(crlfe,string)
End /* Do While eor > 0 */
rec.0 = n
Return
REDIRECT: /* Handle Redirection Header */
s = Pos('Location:',string)
If s = 0 Then
Do
Say msghdr 'Error! Redirection Header Format Invalid!'
Return 16
End
s = s + 10
e = Pos(crlfe,string,s)
If e <= s Then
Do
Say msghdr 'Error! Redirection Header Format Invalid!'
Return 16
End
l = e - s
newloc = SubStr(string,s,l)
Parse Var newloc protocol '://' newsite '/' newpath
newpath = '/'Space(newpath,0)
If newsite = site & newpath = path Then
Do
Say msghdr 'Error! New Site and Path Same as Defaults!'
Return 20
End
If src = 301 Then
mtype = 'Permanently'
Else
mtype = 'Temporarily'
Say msghdr 'Redirected' mtype 'to: ' newsite||newpath
Return 0
SETUP: /* Initialize Conversion Strings */
hexall = XRange('00'x,'FF'x) /* Hex Characters 00-FF */
/* 0 1 2 3 4 5 6 7 8 9 A B C D E F */
ascii = X2C('00010203372D2E2F1605250B0C0D0E0F'||, /* 00-0F */
'101112133C3D322618193F271C1D1E1F'||, /* 10-1F */
'405A7F7B5B6C507D4D5D5C4E6B604B61'||, /* 20-2F */
'F0F1F2F3F4F5F6F7F8F97A5E4C7E6E6F'||, /* 30-3F */
'7CC1C2C3C4C5C6C7C8C9D1D2D3D4D5D6'||, /* 40-4F */
'D7D8D9E2E3E4E5E6E7E8E9ADE0BD5F6D'||, /* 50-5F */
'79818283848586878889919293949596'||, /* 60-6F */
'979899A2A3A4A5A6A7A8A9C04FD0A107'||, /* 70-7F */
'00010203372D2E2F1605250B0C0D0E0F'||, /* 80-8F */
'101112133C3D322618193F271C1D1E1F'||, /* 90-9F */
'405A7F7B5B6C507D4D5D5C4E6B604B61'||, /* A0-AF */
'F0F1F2F3F4F5F6F7F8F97A5E4C7E6E6F'||, /* B0-BF */
'7CC1C2C3C4C5C6C7C8C9D1D2D3D4D5D6'||, /* C0-CF */
'D7D8D9E2E3E4E5E6E7E8E9ADE0BD5F6D'||, /* D0-DF */
'79818283848586878889919293949596'||, /* E0-EF */
'979899A2A3A4A5A6A7A8A9C04FD0A107') /* F0-FF */
/* 0 1 2 3 4 5 6 7 8 9 A B C D E F */
ebcdic = X2C('000102030009007F0000000B0C0D0E0F'||, /* 00-0F */
'10111200000008001819000000000000'||, /* 10-1F */
'00001C00000A171B0000000000050607'||, /* 20-2F */
'00001600001E0004000000001415001A'||, /* 30-3F */
'20000000000000000000002E3C282B7C'||, /* 40-4F */
'2600000000000000000021242A293B5E'||, /* 50-5F */
'2D2F00000000000000007C2C255F3E3F'||, /* 60-6F */
'000000000000000000603A2340273D22'||, /* 70-7F */
'00616263646566676869000000000000'||, /* 80-8F */
'006A6B6C6D6E6F707172000000000000'||, /* 90-9F */
'007E737475767778797A0000005B0000'||, /* A0-AF */
'000000000000000000000000005D0000'||, /* B0-BF */
'7B414243444546474849000000000000'||, /* C0-CF */
'7D4A4B4C4D4E4F505152000000000000'||, /* D0-DF */
'5C00535455565758595A000000000000'||, /* E0-EF */
'30313233343536373839000000000000') /* F0-FF */
Return
EPILOGUE:
/*-------------------------- SAMPHTTP REXX ---------------------------*/
/* Description: */
/* SAMPHTTP will use REXX TCP/IP Sockets to connect to the HTTP */
/* server and port specified in variable "site". The Server file */
/* will to written to the MVS File pointed to by the OUTPUT DD */
/* statement in the JCL that invokes SAMPHTTP. */
/* */
/* SAMPHTTP can also be invoked on VM/CMS for testing and will */
/* write the Server file to CMS File "OUTPUT DATA" on the supplied */
/* CMS Filemode or your A-Disk if no CMS Filemode is specified.) */
/* */
/* Format: */
/* SAMPHTTP <fm> */
/* */
/* Parameters: */
/* FM */
/* An optional CMS Filemode to write the output file to. This */
/* parameter only applies if SAMPHTTP is invoked on VM/CMS. If */
/* no Filemode is specified, the default is "A". */
/* */
/* Options: */
/* None. */
/* */
/* Examples: */
/* Example 1: JCL to Retrieve the Server file on MVS. */
/* */
/* //DELFILE EXEC PGM=IDCAMS */
/* //SYSPRINT DD SYSOUT=* */
/* //SYSIN DD * */
/* DELETE hlq.server.output.file */
/* IF MAXCC = 8 THEN SET MAXCC = 0 */
/* /* */*/
/* //GETFILE EXEC PGM=IKJEFT01 */
/* //SYSEXEC DD DISP=SHR,DSN=hlq.sysexec.pds */
/* //SYSTSPRT DD SYSOUT=* */
/* //OUTPUT DD DSN=hlq.server.output.file, */
/* // DISP=(,CATLG,DELETE),UNIT=SYSDA, */
/* // AVGREC=K,SPACE=(nnn,(1,1),RLSE), */
/* // RECFM=FB,LRECL=nnn */
/* //SYSTSIN DD * */
/* %SAMPHTTP */
/* /* */*/
/* */
/* Example 2: Retrieve Server File to CMS A-Disk. */
/* */
/* SAMPHTTP */
/* */
/* Example 3: Retrieve Server File to CMS D-Disk. */
/* */
/* SAMPHTTP D */
/* */
/* Messages: */
/* SAMPHTTP: Creating Socket Set... */
/* SAMPHTTP: Opening Socket... */
/* SAMPHTTP: Connecting to www.yourserver.com... */
/* SAMPHTTP: Sending Request to www.yoursrever.com... */
/* SAMPHTTP: Receiving Data from www.yourserver.com... */
/* SAMPHTTP: Closing Socket... */
/* SAMPHTTP: Deleting Socket Set... */
/* */
/* SAMPHTTP: Redirected Permanently to: xxx.xxxx/yyyyy/zzzz */
/* SAMPHTTP: Redirected Temporarily to: xxx.xxxx/yyyyy/zzzz */
/* */
/* SAMPHTTP: Initialization Error nn: sockid, maxdesc, service */
/* SAMPHTTP: Socket Error nn: socket */
/* SAMPHTTP: Connect Error nn: www.yourserver.com */
/* SAMPHTTP: Send Error nn: www.yourserver.com */
/* */
/* SAMPHTTP: Invalid Operating Environment: xxx (RC=12) */
/* SAMPHTTP: SAMPHTTP must be invoked from CMS or TSO Batch. */
/* */
/* SAMPHTTP: Error! Redirection Header Format Invalid! (RC=16) */
/* */
/* SAMPHTTP: Error! New Site and Path Same as Defaults! (RC=20) */
/* */
/* SAMPHTTP: Error nn Opening MVS File OUTPUT (RC=nn) */
/* SAMPHTTP: The DD Statement for OUTPUT is Probably Missing */
/* */
/* SAMPHTTP: Error nn No R/W x-Disk Available! (RC=nn) */
/* */
/* SAMPHTTP: Creating MVS File "OUTPUT"... */
/* */
/* SAMPHTTP: Error nn Creating MVS File "OUTPUT". (RC=nn) */
/* */
/* SAMPHTTP: Creating CMS File "OUTPUT DATA x"... */
/* */
/* SAMPHTTP: Error nn Creating CMS File "OUTPUT DATA x". (RC=nn) */
/* */
/* Return Codes: */
/* RC = 0 */
/* Indicates that the Exec completed normally. */
/* RC = 12 */
/* Indicates that an Invalid Operating Environment was Found. */
/* RC = 16 */
/* Indicates that a Redirection Header was Returned, but the */
/* Format of the Header is Invalid. */
/* RC = 20 */
/* Indicates that a Redirection Header was Returned, but the */
/* New Site and Path in the Header are the same as the Default */
/* Site and Path used in the Exec. */
/* RC = nn */
/* Indicates that an error occurred while executing a command. */
/* */
/* Updates: */
/* 2015-06-11 (Dale R. Smith) */
/* Initial Version Created. */
/* (Sample Code based on Production Code written in 2009.) */
/*--------------------------------------------------------------------*/
--
Dale R. Smith
>
>----------------------------------------------------------------------
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to [email protected] with the message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
