Re: Rexx Concurrency (was: Sorting CSV data ...)
Ok. Thanks. John T. Abell President International Software Products Tel: 800-295-7608 Ext: 224 International: 1-416-593-5578 Ext: 224 Fax: 800-295-7609 International: 1-416-593-5579 E-mail: john.ab...@intnlsoftwareproducts.com Web: www.ispinfo.com This email may contain confidential and privileged material for the sole use of the intended recipient(s). Any review, use, retention, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive on behalf of the named recipient), please contact the sender by reply email and delete all copies of this message. Also,email is susceptible to data corruption, interception, tampering, unauthorized amendment and viruses. We only send and receive emails on the basis that we are not liable for any such corruption, interception, tampering, amendment or viruses or any consequence thereof. -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Sankaranarayanan, Vignesh Sent: Monday, April 28, 2014 10:31 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: Rexx Concurrency (was: Sorting CSV data ...) > In your case, you estimated 5000 pings and SNMP GET's. If you spawn that many process under Netview could be detrimental. Can SNMP become CPU intensive? Will there be a situation where you need to cancel 5000 processes (or worse - restart Netview because of the overload). Well, 5000 printers but if I hadn't mentioned before, apologies - of the 5000, I'm interested in only around 700. Whenever I encounter a printer that belongs to the 4300 pool, I just ITERATE in REXX. And even for the 700, I don't SNMP it all. I ping them first; if the ping fails, nothing else. If ping is okay, get more info via SNMP. So the true number maybe near 550. I haven't bothered checking how many pings fail (yet). NetView does run with higher priority, so I do understand the risks involved. > Will the number increase from 5000 to a much larger number? Not likely. Maybe 7000, but that's assuming the worst case where we just need those printers. > Will someone decide running this every 10 minutes is a great idea? I'm the only one who runs it. No one is going to automate it without my knowledge. - Vignesh Mainframe Admin -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Jon Perryman Sent: 27 April 2014 22:27 To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: Rexx Concurrency (was: Sorting CSV data ...) There is often a reason why z/OS makes simple UNIX tasks difficult or makes good people do bad things. In your case, you estimated 5000 pings and SNMP GET's. If you spawn that many process under Netview could be detrimental. Can SNMP become CPU intensive? Will there be a situation where you need to cancel 5000 processes (or worse - restart Netview because of the overload). Will the number increase from 5000 to a much larger number? Will someone decide running this every 10 minutes is a great idea? Automation products run at a high dispatching priority and run mission critical commands. You can potentially take away valuable CPU from much more important workload or even worse, cause a critical command to be delayed. Submitting JCL is the best solution but submitting 1000 jobs to be a bit much.Submitting 3 or 4 is more reasonable but you will need to use a queue to pull the next IP address. For the queue, you could install / use the REXX interface to MQ or DB2. Alternatively you could install / use RXVSAM from CBT and use a VSAM file as a queue. If you do this in Netview, I suggest scheduling multiple Netview commands that are processed by a set of specific Netview command tasks. You can stop / start these tasks if Netview get's overwhelmed and Netview cancel command is available to cancel the commands. Jon Perryman. - Original Message - > From: "Sankaranarayanan, Vignesh" > > > Continuing dream mode, is it possible to SNMP GET and PING from inside > OMVS, whilst maintaining the NetView REXX as parent? > That would allow me to spin up threads I hope ... > > -Original Message- > Of Paul Gilmartin > > On Thu, 17 Apr 2014 17:41:12 -0500, Norbert Friemel wrote: > >> On Thu, 17 Apr 2014 19:26:17 +, Sankaranarayanan, Vignesh wrote: >> >>> Ok.. Is it possible to fire off other REXXes repetitively (let's say > 2 or 3, each doing one function) that will not RETURN to the main, but > write their output to a dataset once done? >> >> Submit batch jobs (PGM=IKJEFT01,PARM='%rxsub?? p1 p2 p3) from the main? >> > Neither DDLIST nor Google gives me relevant hits for RXSUB. > > (But it really shouldn't be that hard. For each job: > > RC = BPXWDYN( 'alloc rtddn(RDR) sysout writer(INTRDR) m
Re: Rexx Concurrency (was: Sorting CSV data ...)
> In your case, you estimated 5000 pings and SNMP GET's. If you spawn that many
> process under Netview could be detrimental. Can SNMP become CPU intensive?
> Will there be a situation where you need to cancel 5000 processes (or worse -
> restart Netview because of the overload).
Well, 5000 printers but if I hadn't mentioned before, apologies - of the 5000,
I'm interested in only around 700. Whenever I encounter a printer that belongs
to the 4300 pool, I just ITERATE in REXX. And even for the 700, I don't SNMP it
all. I ping them first; if the ping fails, nothing else. If ping is okay, get
more info via SNMP. So the true number maybe near 550. I haven't bothered
checking how many pings fail (yet).
NetView does run with higher priority, so I do understand the risks involved.
> Will the number increase from 5000 to a much larger number?
Not likely. Maybe 7000, but that's assuming the worst case where we just need
those printers.
> Will someone decide running this every 10 minutes is a great idea?
I'm the only one who runs it. No one is going to automate it without my
knowledge.
- Vignesh
Mainframe Admin
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf
Of Jon Perryman
Sent: 27 April 2014 22:27
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Rexx Concurrency (was: Sorting CSV data ...)
There is often a reason why z/OS makes simple UNIX tasks difficult or makes
good people do bad things. In your case, you estimated 5000 pings and SNMP
GET's. If you spawn that many process under Netview could be detrimental. Can
SNMP become CPU intensive? Will there be a situation where you need to cancel
5000 processes (or worse - restart Netview because of the overload). Will the
number increase from 5000 to a much larger number? Will someone decide running
this every 10 minutes is a great idea?
Automation products run at a high dispatching priority and run mission critical
commands. You can potentially take away valuable CPU from much more important
workload or even worse, cause a critical command to be delayed.
Submitting JCL is the best solution but submitting 1000 jobs to be a bit
much.Submitting 3 or 4 is more reasonable but you will need to use a queue to
pull the next IP address. For the queue, you could install / use the REXX
interface to MQ or DB2. Alternatively you could install / use RXVSAM from CBT
and use a VSAM file as a queue.
If you do this in Netview, I suggest scheduling multiple Netview commands that
are processed by a set of specific Netview command tasks. You can stop / start
these tasks if Netview get's overwhelmed and Netview cancel command is
available to cancel the commands.
Jon Perryman.
- Original Message -
> From: "Sankaranarayanan, Vignesh"
>
>
> Continuing dream mode, is it possible to SNMP GET and PING from inside
> OMVS, whilst maintaining the NetView REXX as parent?
> That would allow me to spin up threads I hope ...
>
> -Original Message-
> Of Paul Gilmartin
>
> On Thu, 17 Apr 2014 17:41:12 -0500, Norbert Friemel wrote:
>
>> On Thu, 17 Apr 2014 19:26:17 +, Sankaranarayanan, Vignesh wrote:
>>
>>> Ok.. Is it possible to fire off other REXXes repetitively (let's say
> 2 or 3, each doing one function) that will not RETURN to the main, but
> write their output to a dataset once done?
>>
>> Submit batch jobs (PGM=IKJEFT01,PARM='%rxsub?? p1 p2 p3) from the main?
>>
> Neither DDLIST nor Google gives me relevant hits for RXSUB.
>
> (But it really shouldn't be that hard. For each job:
>
> RC = BPXWDYN( 'alloc rtddn(RDR) sysout writer(INTRDR) msg(WTP)' )
> /* write JCL to RDR with EXECIO. */
> RC = BPXWDYN( 'free dd('RDR') msg(WTP)' )
>
> The hard part is detecting completion and getting status back.)
>
> -- gil
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
MARKSANDSPENCER.COM
Unless otherwise stated above:
Marks and Spencer plc
Registered Office:
Waterside House
35 North Wharf Road
London
W2 1NW
Registered No. 214436 in England and Wales.
Telephone (020) 7935 4422
Facsimile (020) 7487 2670
www.marksandspencer.com
Please note that electronic mail may be monitored.
This e-mail is confidential. If you received it by mistake, please let us know
and then delete it from your system; you should not copy, disclose, or
distribute its contents to anyone nor act in reliance on this e-mail, as this
is prohibited and may be unlawful.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Rexx Concurrency (was: Sorting CSV data ...)
On Sun, 27 Apr 2014 14:26:51 -0700, Jon Perryman wrote: > >Submitting JCL is the best solution but submitting 1000 jobs to be a bit >much.Submitting 3 or 4 is more reasonable but you will need to use a queue to >pull the next IP address. For the queue, you could install / use the REXX >interface to MQ or DB2. Alternatively you could install / use RXVSAM from CBT >and use a VSAM file as a queue. > I have used POSIX pipes to control a queue in a POSIX shell script. Shell is more suited to concurrent operations than Rexx; simply it has the "&" construct. In my case I wanted concurrency but conservatively (arbitrarily) limited to 5 concurrent processes. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Rexx Concurrency (was: Sorting CSV data ...)
There is often a reason why z/OS makes simple UNIX tasks difficult or makes
good people do bad things. In your case, you estimated 5000 pings and SNMP
GET's. If you spawn that many process under Netview could be detrimental. Can
SNMP become CPU intensive? Will there be a situation where you need to cancel
5000 processes (or worse - restart Netview because of the overload). Will the
number increase from 5000 to a much larger number? Will someone decide running
this every 10 minutes is a great idea?
Automation products run at a high dispatching priority and run mission critical
commands. You can potentially take away valuable CPU from much more important
workload or even worse, cause a critical command to be delayed.
Submitting JCL is the best solution but submitting 1000 jobs to be a bit
much.Submitting 3 or 4 is more reasonable but you will need to use a queue to
pull the next IP address. For the queue, you could install / use the REXX
interface to MQ or DB2. Alternatively you could install / use RXVSAM from CBT
and use a VSAM file as a queue.
If you do this in Netview, I suggest scheduling multiple Netview commands that
are processed by a set of specific Netview command tasks. You can stop / start
these tasks if Netview get's overwhelmed and Netview cancel command is
available to cancel the commands.
Jon Perryman.
- Original Message -
> From: "Sankaranarayanan, Vignesh"
>
>
> Continuing dream mode, is it possible to SNMP GET and PING from inside OMVS,
> whilst maintaining the NetView REXX as parent?
> That would allow me to spin up threads I hope ...
>
> -Original Message-
> Of Paul Gilmartin
>
> On Thu, 17 Apr 2014 17:41:12 -0500, Norbert Friemel wrote:
>
>> On Thu, 17 Apr 2014 19:26:17 +, Sankaranarayanan, Vignesh wrote:
>>
>>> Ok.. Is it possible to fire off other REXXes repetitively (let's say
> 2 or 3, each doing one function) that will not RETURN to the main, but write
> their output to a dataset once done?
>>
>> Submit batch jobs (PGM=IKJEFT01,PARM='%rxsub?? p1 p2 p3) from the main?
>>
> Neither DDLIST nor Google gives me relevant hits for RXSUB.
>
> (But it really shouldn't be that hard. For each job:
>
> RC = BPXWDYN( 'alloc rtddn(RDR) sysout writer(INTRDR) msg(WTP)' )
> /* write JCL to RDR with EXECIO. */
> RC = BPXWDYN( 'free dd('RDR') msg(WTP)' )
>
> The hard part is detecting completion and getting status back.)
>
> -- gil
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Rexx Concurrency (was: Sorting CSV data ...)
On 4/17/2014 4:09 PM, Sankaranarayanan, Vignesh wrote: Continuing dream mode, is it possible to SNMP GET and PING from inside OMVS, whilst maintaining the NetView REXX as parent? That would allow me to spin up threads I hope ... You can do a lot of neat z/OS UNIX-ish things in REXX using bpxwunix() which should work under Netview. -- Edward E Jaffe Phoenix Software International, Inc 831 Parkview Drive North El Segundo, CA 90245 http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Rexx Concurrency (was: Sorting CSV data ...)
Continuing dream mode, is it possible to SNMP GET and PING from inside OMVS,
whilst maintaining the NetView REXX as parent?
That would allow me to spin up threads I hope ...
- Vignesh
Mainframe Admin
-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf
Of Paul Gilmartin
Sent: 18 April 2014 00:02
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Sorting CSV data that begins with an IP address
On Thu, 17 Apr 2014 17:41:12 -0500, Norbert Friemel wrote:
>On Thu, 17 Apr 2014 19:26:17 +, Sankaranarayanan, Vignesh wrote:
>
>>Ok.. Is it possible to fire off other REXXes repetitively (let's say 2 or 3,
>>each doing one function) that will not RETURN to the main, but write their
>>output to a dataset once done?
>
>Submit batch jobs (PGM=IKJEFT01,PARM='%rxsub?? p1 p2 p3) from the main?
>
Neither DDLIST nor Google gives me relevant hits for RXSUB.
(But it really shouldn't be that hard. For each job:
RC = BPXWDYN( 'alloc rtddn(RDR) sysout writer(INTRDR) msg(WTP)' )
/* write JCL to RDR with EXECIO. */
RC = BPXWDYN( 'free dd('RDR') msg(WTP)' )
The hard part is detecting completion and getting status back.)
-- gil
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
MARKSANDSPENCER.COM
Unless otherwise stated above:
Marks and Spencer plc
Registered Office:
Waterside House
35 North Wharf Road
London
W2 1NW
Registered No. 214436 in England and Wales.
Telephone (020) 7935 4422
Facsimile (020) 7487 2670
www.marksandspencer.com
Please note that electronic mail may be monitored.
This e-mail is confidential. If you received it by mistake, please let us know
and then delete it from your system; you should not copy, disclose, or
distribute its contents to anyone nor act in reliance on this e-mail, as this
is prohibited and may be unlawful.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Rexx Concurrency (was: Sorting CSV data ...)
Thanks. I haven't used it before. Let me look it up .. - Vignesh Mainframe Admin -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Paul Gilmartin Sent: 17 April 2014 20:55 To: IBM-MAIN@LISTSERV.UA.EDU Subject: Rexx Concurrency (was: Sorting CSV data ...) On Thu, 17 Apr 2014 18:58:10 +, Sankaranarayanan, Vignesh wrote: > >I'm looping some 5000 times in REXX and doing functions (NetView ping, SNMP >walk, etc) sequentially. They don't necessarily need to be sequential. I'm >just going through a list of printers and I want to test them. > >So.. Is it possible that I make the first run a data run - reading parameters >from files for each printer (reading PDS member) - and the second run as a >thread creator of sorts. Each thread going off to test one printer. > >The first run should be almost instantaneous. My thinking is.. Is it possible >to parallelly do the pinging and SNMP connection bits. It would cut 20 minutes >runtime to just a couple. > I suggested elsewhere in this thread, "SYSCALL spawn". The only z/OS-supported Rexx facility that provides concurrency (AFAIK). Considerable coding; much to learn if you haven't used it before; but it's effective. Each child (I don't believe you can do 5000 at once), could write its report to a separate file, and when all complete the master could merge, catenate, or summarize them. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN MARKSANDSPENCER.COM Unless otherwise stated above: Marks and Spencer plc Registered Office: Waterside House 35 North Wharf Road London W2 1NW Registered No. 214436 in England and Wales. Telephone (020) 7935 4422 Facsimile (020) 7487 2670 www.marksandspencer.com Please note that electronic mail may be monitored. This e-mail is confidential. If you received it by mistake, please let us know and then delete it from your system; you should not copy, disclose, or distribute its contents to anyone nor act in reliance on this e-mail, as this is prohibited and may be unlawful. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Rexx Concurrency (was: Sorting CSV data ...)
On Thu, 17 Apr 2014 18:58:10 +, Sankaranarayanan, Vignesh wrote: > >I'm looping some 5000 times in REXX and doing functions (NetView ping, SNMP >walk, etc) sequentially. They don't necessarily need to be sequential. I'm >just going through a list of printers and I want to test them. > >So.. Is it possible that I make the first run a data run - reading parameters >from files for each printer (reading PDS member) - and the second run as a >thread creator of sorts. Each thread going off to test one printer. > >The first run should be almost instantaneous. My thinking is.. Is it possible >to parallelly do the pinging and SNMP connection bits. It would cut 20 minutes >runtime to just a couple. > I suggested elsewhere in this thread, "SYSCALL spawn". The only z/OS-supported Rexx facility that provides concurrency (AFAIK). Considerable coding; much to learn if you haven't used it before; but it's effective. Each child (I don't believe you can do 5000 at once), could write its report to a separate file, and when all complete the master could merge, catenate, or summarize them. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
