It's called a producer consumer pattern. It's directly applicable here - the producer creates the work and the consumer(s) process it.
Consumers can either be in separate address spaces or attached subtasks. It doesn't matter for purposes of what they are doing. You - as it says below - link them with a queue. You can use MQ, or USS queues (forget what they are called - it's been a while since I wrote anything using USS) to do this if you don't want to write your own queueing implementation. MQ has the advantage that it handles a lot of the concurrency concerns. You can hang multiple consumers off a queue and it will post all the waiting ECBs when input arrives. The first one to wake up and start work will get the message and carries on, the rest should just loop back to wait. We've used this in this manner in a few places. I don't have access to TSO from where I am or I'd post the control loop. ________________________________________ From: IBM Mainframe Assembler List [[email protected]] on behalf of Farley, Peter x23353 [[email protected]] Sent: Thursday, July 24, 2014 10:43 AM To: [email protected] Subject: Re: Subtasking program Tony, I am also not aware of any specific examples out in the wild. However, you may have some luck examining the area that the CS kids call "design patterns". These "patterns" mostly seem to be applied to object-oriented programming, but the principles are the same for z/OS subtasking, just translate what they call "thread" to "subtask" and the paradigms pretty much still apply. Here is one paradigm I have used successfully Feeder (sub)task: Wait for input (and maybe a normal termination ECB as well) Receive input Lock the queue Add item to be processed to the queue (usually FIFO, YMMV) Unlock the queue POST ECB's for *all* reader tasks if they are not already posted (PoOP has a good example of this kind of POST in the Instruction Examples appendix) Loop to wait for new input Eater (sub)tasks(s): Wait on unique ECB posted by Feeder (and maybe a normal termination ECB as well) Clear the ECB Lock the queue Pop first item off the queue (or last, or whatever queue processing you need to get one item) Unlock the queue Process the item Loop to wait for more input "Lock the queue" itself can have another WAIT (ECB or timed WAIT) embedded for the queue to be available, but there are some good queue lock management algorithms out there, and IIRC PoOP has some good queue serialization examples in the Instruction Examples appendix. You also need initialization logic in the eater (sub)tasks(s) to wait until the feeder subtask finishes setting up the initial queue structure. The feeder would post all eater ECB's once after initialization so they can finish their own initializations after the queue is initialized. The feeder (sub)task termination logic (normal or abnormal) needs to decide what to do if there are un-processed items left in the queue at termination. Similarly the eater (sub)task abnormal termination logic needs to decide if it should (or can) return the partially processed item to the queue and what to do with the item if it cannot be returned to the queue. HTH Peter -----Original Message----- From: IBM Mainframe Assembler List [mailto:[email protected]] On Behalf Of Tony Thigpen Sent: Thursday, July 24, 2014 8:51 AM To: [email protected] Subject: Re: Subtasking program > May be faster to have the main build a queue of records and let the subtasks > feed themselves. There are interesting serialization concerns with multiple > eaters. The feeder can post all waiting tasks when food is available. I know this is a very old thread... I have been re-reading it simply because I am "looking for a better way". Does anybody know of any real good documentation on subtask usage as it relates to, what Binyamin called, "single feeder, multiple eater" processing? Including the queuing of requests. Or examples. I have scanned the CBT tape, but I did not spot any that indicated they used such. Tony Thigpen -----Original Message ----- From: Binyamin Dissen Sent: 04/10/2011 11:51 AM > On Sun, 10 Apr 2011 06:23:15 -0400 Joe Owens <[email protected]> > wrote: > > :>After such a great response to my last post, I thought I would try here > :>again for some useful advice on the best design for a program I am about to > :>write. > :>I have a mainline program that reads an input file and will then pass a > :>record to a subtask to process. > > May be faster to have the main build a queue of records and let the subtasks > feed themselves. There are interesting serialization concerns with multiple > eaters. The feeder can post all waiting tasks when food is available. > > :>The subtask has a work cycle > :>MQPUT messsage > :>MQGET message (with wait option, by corellid, gets the response message) > :>write message to preopened output file. > :>When the mainline program wishes to send another message, it must choose a > :>non-busy subtask, or attach another. The whole point is to drive a certain > :>message rate, so I do not wish to queue more than 1 piece of work to any > :>subtask at a time > > The attach has some overhead. Depending on the processing speed, it may take > more time to do the attach than let an existing task become ready. > > :>The subtask cycle will take about 150ms. I would like to be able to process > :>up to 300 messages per second, although normal rates would be much lower. > :>So questions; > :>Use pause/release or wait/post? > > Pasue/release is probably slightly faster. > > :>For the output file write, what would be the best way of serialising the > :>file writes? > > As the others have suggested, a queue for the main task, where the subtasks > are feeders. > > :>each subtask wraps an enq/deq around the put? > > Yuck. > > :>Another subtask has a work queue of records to be written which is added to > :>by the other tasks? In this case what is the best scheme for dispatch of > :>this task, when many others may wish to give him work? > > Use the main task. And the feeders should post its ECB if it is waiting. > > :>Any other comments or suggestions would be appreciated too. > > What to do if a subtask is non-responsive, i.e., does not complete within the > expected time? > > What to do if a subtask abends handling a request? > > What if MQ goes down in the middle? > > -- > Binyamin Dissen <[email protected]> > http://www.dissensoftware.com > > Director, Dissen Software, Bar & Grill - Israel > > > Should you use the mailblocks package and expect a response from me, > you should preauthorize the dissensoftware.com domain. > > I very rarely bother responding to challenge/response systems, > especially those from irresponsible companies. > > This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
